Meta Platforms is facing employee unrest as it cuts jobs and expands its AI unit, with workers expressing dissatisfaction over new roles and intrusive monitoring practices.
Oncological genomic sequencing cross-correlated with clinical patient outcome data to improve cancer treatment targeting based on real world patient outcome data. Personally, I work on data pipeline stuff. The mission is fantastic; the management and C-suites are a massive pain in my ass. But when are they not?
I hope pipeline stuff is better than when I started. I (not so) accidentally played a big part in revolutionizing my field by building anndata and scanpy. Then I tried doing it again by writing sequencing stuff in Rust, but was too early / too bad, but now that’s happening too!
I’m personally playing with the idea of putting together a POC to transition our “foundation” data warehouse from PSQL to a graphDB, because the extensibility and maintainability of our current system is fucking awful. Like, some upstream entity gets a version bump and there’s like 5 systems we have to go through and add columns to various tables (usually 2-4 such modifications in each workflow - and it doesn’t help how my manager fucking loves to slice work up so much that it becomes a massive pain to integrate, instead of telling one Eng “integrate this new data element” and have it done, soup to nuts, in a week or two across the whole ecosystem) and occasionally fuck around with joins and so on every single time there’s a new piece of data we want to integrate. And we have no capability to scan back historically and evaluate our holistic state at some particular time index, which can be really helpful for some applications.
Anyways, I’m fucking swamped at work so haven’t touched that at all, but I’ve wanted to explore that idea for well over a year and a half at this point.
I have little experience with graphdb, but a lot of experience with the pain you’re describing. Maintaining schemas is a pain, maybe if you don’t need the performance, you can go that route!
The thing that interests me about it is that it will be a lot more trivially interrogable by ML stuff (bespoke ML specifically, not LLM), which could glean an absolute shitload of interesting insights for us.
I am an enormous fucking Luddite for a whole swath of reasons when it comes to LLMs, but ML outside of that context can be immensity powerful when employed correctly.
If you’re doing PSQL (or any typical relational DB flavor), there’s a lot more complexity in terms of understanding the shape of the data, what joins to what, how to optimize queries, etc. Graph DBs are gonna be easier for a model to explore, since they can just do stuff like “I want to see tests with samples that have reactivity to mutation ABC on chromosome 14 over a threshold of X”, which is a lot easier for an ML agent (or less experienced developer, or even a molecular biologist with limited CS/DB experience) to just intuitively evaluate correctly using the syntax of GraphQL than it would be trying to do a shitload of joins between 6 or 7 tables in PSQL.
I agree, I left software dev 10 years ago. The previous 3 decades were a golden age - we went from working with wild-west code and flat files to modular programming and databases, then object-oriented programming, then the web came along and brought another wild-west period, then came millions of packages and frameworks du jour. A giant wave of all that was just curling overhead when I left, but hadn’t crashed down yet. Now if your web page with a button on it doesn’t use 47 libraries you’re not a software engineer lol.
Honestly, I’m glad I didn’t get sucked into the big tech vortex a decade or so ago. Much happier working in biotech these days.
What are you doing? I’m with scverse, doing single cell stuff.
Oncological genomic sequencing cross-correlated with clinical patient outcome data to improve cancer treatment targeting based on real world patient outcome data. Personally, I work on data pipeline stuff. The mission is fantastic; the management and C-suites are a massive pain in my ass. But when are they not?
I hope pipeline stuff is better than when I started. I (not so) accidentally played a big part in revolutionizing my field by building anndata and scanpy. Then I tried doing it again by writing sequencing stuff in Rust, but was too early / too bad, but now that’s happening too!
I’m personally playing with the idea of putting together a POC to transition our “foundation” data warehouse from PSQL to a graphDB, because the extensibility and maintainability of our current system is fucking awful. Like, some upstream entity gets a version bump and there’s like 5 systems we have to go through and add columns to various tables (usually 2-4 such modifications in each workflow - and it doesn’t help how my manager fucking loves to slice work up so much that it becomes a massive pain to integrate, instead of telling one Eng “integrate this new data element” and have it done, soup to nuts, in a week or two across the whole ecosystem) and occasionally fuck around with joins and so on every single time there’s a new piece of data we want to integrate. And we have no capability to scan back historically and evaluate our holistic state at some particular time index, which can be really helpful for some applications.
Anyways, I’m fucking swamped at work so haven’t touched that at all, but I’ve wanted to explore that idea for well over a year and a half at this point.
I have little experience with graphdb, but a lot of experience with the pain you’re describing. Maintaining schemas is a pain, maybe if you don’t need the performance, you can go that route!
The thing that interests me about it is that it will be a lot more trivially interrogable by ML stuff (bespoke ML specifically, not LLM), which could glean an absolute shitload of interesting insights for us.
I am an enormous fucking Luddite for a whole swath of reasons when it comes to LLMs, but ML outside of that context can be immensity powerful when employed correctly.
For sure, my lab has been doing that for a long time.
How is graphdb more ML-friendly?
If you’re doing PSQL (or any typical relational DB flavor), there’s a lot more complexity in terms of understanding the shape of the data, what joins to what, how to optimize queries, etc. Graph DBs are gonna be easier for a model to explore, since they can just do stuff like “I want to see tests with samples that have reactivity to mutation ABC on chromosome 14 over a threshold of X”, which is a lot easier for an ML agent (or less experienced developer, or even a molecular biologist with limited CS/DB experience) to just intuitively evaluate correctly using the syntax of GraphQL than it would be trying to do a shitload of joins between 6 or 7 tables in PSQL.
I agree, I left software dev 10 years ago. The previous 3 decades were a golden age - we went from working with wild-west code and flat files to modular programming and databases, then object-oriented programming, then the web came along and brought another wild-west period, then came millions of packages and frameworks du jour. A giant wave of all that was just curling overhead when I left, but hadn’t crashed down yet. Now if your web page with a button on it doesn’t use 47 libraries you’re not a software engineer lol.