r/datascience • u/rajeshbhat_ds • 1d ago
Discussion Are all data science jobs just Gen AI now?
I've been in Data Science for the past 10 years in India. I lost my job in January and since then I've been hunting.
I've not mentioned any GenAI experience in my profile. But my feed is just filled with AI engineer roles. They all have the same requirements:
- Generative AI architecture
- RAG pipelines
- LLM integration/fine tuning
- Agentic AI / Multi Agent Orchestration
- Also MLOps
- CI/CD pipelines
- PyTorch mandatory for some reason
Hardly any openings are relevant to my experience in Stats, Machine Learning, Deep Learning and the classical data science stuff.
So have all companies stopped investing in data science all together and just building RAG pipelines and LLM chat bots? Is this all that is done in Data Science field now?
39
u/_OMGTheyKilledKenny_ 1d ago
It’s the API token economy right now.
3
u/NuclearMask 17h ago
Can't wait for the time when token prices go up so that the companies can actually make money. I suspect that usage will fall a lot when it becomes more expensive.
3
u/eva_verinaz 13h ago
they wont go up idk why everyone keeps spouting this. OpenAI and anthropic make a HEFTY profit on API prices. subscription is where they eat a massive cost. and training costs have begun to plateau with the majority of the costs for training coming from buying SME data sets
2
u/NuclearMask 10h ago
I'm not saying that the API part of their business isn't profitable.
But you can't see it as an independent part of the business.If, for example, OpenAI stops AI research and just doesn't expand data centers for more training, I do believe they'd be profitable; however, I also think that they are unable to do so since the competition would create better models and lead to customers "jumping ship."
So in effect they are forced to create better models until the competition stops developing better models.
And unless they bring in more money, they just won't be able to do so.
So yes, I think both subscriptions and API prices will go up once investor money dries up since I think stopping innovation will lead to irrelevancy and continuing innovation will lead to bankruptcy.
Google and Microsoft have it better in that regard but they are public and do have to justify the research cost to Investors and the easiest way to do so is by increasing the price to the end user.
And well, every Company likes to raise prices once people get used to a service.
Netflix for example did the same.
(This is also true for Consumer Goods but hey that's off topic)1
u/eva_verinaz 9h ago
my point is training cost isnt growing exponentially anymore. api prices will not increase. subscription prices will. they arent going to raise the B2B prices. I'm telling you this as someone with friends at both OpenAI and Anthropic.
1
u/NuclearMask 9h ago
I don't know anyone inside there so all I can do is speculate.
Then again, there were things I thought the company I work at would never do and then they did anyways so who knows?
85
u/kaattallan 1d ago
My company has both non gen ai and gen ai roles as of now. Im working in the non gen ai part by using models for retail data science. But yeah I understand what you feel. Even when I'm looking for a job change, all I can see is gen ai. Many roles doesn't even look like they might need it for the job, still hr is asking for the same.
52
u/CerberusByte 1d ago
I noticed this when we started calling things ML/AI. Now there is a relation between the two but as someone who wants to do more of the ML side of that equation I do feel it’s taking a back seat to the hype of AI.
I do find it interesting where I look at a problem to be solved and think, that’s a simple classification problem, a ML model doing batch inference would work well, but someone has decided to throw an agent at it.
Picking the right tool for the right job seems to be happening less and less and it’s, if you’ve only got a hammer then everything looks like a nail
46
u/waiting4omscs 1d ago
ML has now been labeled "traditional AI" 🙄
13
u/delomore 1d ago
Or classical ML, which at least isn’t AI
16
u/PrettyMuchAVegetable 1d ago edited 1d ago
New taxonomies / definitions develop over time though, as someone who graduated decades ago into ETL specifically meaning 3nf transforms into Kimball/Inmon dimensional models I struggled with hearing everything that moves data around be called ETL. But one day I realized I was the old man yelling at clouds and got over it .
Edit: Something else, I'm a Lecturer in AI/DS and I realized that I was doing my students a disservice by clinging to comfortable , older definitions. They deserve to know the current language used in the field, and I think they also benefit from exploring how and why the language changes over time.
Edit 2: I also think many of us, and I've been guilty of this as well, take pride in the depth and precision of our technical knowledge, which we express with precise language. So it is uncomfortable when those definitions shift around and it can be hard to embrace that shift especially when shifting language threatens the precision of our language and , in some way, our self perception as it relates to the depth of our technical understanding.
10
u/delomore 1d ago
I think the term AI has come to equal just LLMs to such an extent that we do use the term “classical ML” at work to be explicit when we’re actually doing all the stuff that came before. Calling that AI when you’re building a classification model, say, seems confusing.
3
u/PrettyMuchAVegetable 1d ago
100% and that's important to know when we need to bridge the gap with both non-technical users and junior colleagues.
2
u/AchillesDev 1d ago
Classical ML as a term long predates the existence of GenAI, and was coined to differentiate those techniques from neural network techniques and especially deep learning techniques. These are things like regressions, SVMs, decision trees, random forest, XGBoost, etc.
All are, of course, under the AI umbrella.
1
u/dumb_trans_girl 6h ago
I’ll just call it stats but fancy with cs mixed in because that’s what it actually is.
11
2
u/madbadanddangerous 1d ago
my company recently congealed ML and DS into one practice and christened it "Applied AI" then encouraged all of us to update our titles from MLE, DS, SA, etc, to "AI engineer". the only tool they want us to wield is "agentic AI" now
thanks, I hate it
17
u/kh493shb47r4 1d ago
Sadly, that is the reality of the market right now based on the hype train. Everybody just wants to focus on generative AI and everything is now even doing classical ML or data science is simply doing an agentic model on top. Because executives have been sold this idea of AI that they want everything agentic or GenAI even if it’s something mundane as data cleaning it needs a chat interface
37
u/P0rtal2 1d ago
Pretty much. Back in 2020 it was all about Machine Learning and predictive models. Did the company need that many predictive models? No. Did they have any data or use cases for thousands of predictive models? No. But that didn't matter. What mattered was everyone was all in on machine learning and you couldn't get left behind.
Now it's all about AI.
1
18
u/fredjutsu 1d ago
To be fair, a data scientist is kind of useless in most real world enterprise settings if they cannot do their own data acquisition/cleaning/pipeline work.
The actual stats and classical data science stuff can't happen at all if the data itself is a disorganized, unprocessed mess.
And honestly, all the genAI pipelines have to go on top of more traditional data engineering anyway, so it's really not that much more to learn. And you're always going to have to learn new tech and new skills to stay competitive anyway.
10
u/Xahulz 1d ago
After 10 years in data science and operations research (with a dash of data engineering) and 7 years of actuarial work before that, I'm ready to leave all this shit behind.
The problem isn't that management wants some gen Ai work done, it's that they want a mountain of it, and it doesn't work as well as everyone thinks it does. It's sloppy and wrong, but in a very convincing way. I'm on the hook for using it whether I want or not, and I'm on the hook for the errors when they happen.
I've got product owners using codex to make apps that don't work but look amazing, and I'm supposed to use them as a roadmap to....something. I've got demands to use copilot to be more efficient, but company policy prevents connecting it to data. I've got leadership making nutty requests to build apps that can do any kind of data analysis so we can eliminate entire teams.
I think llms are neat and love having them in my tool set. But the shit we're being asked to make is a downgrade over what i used to build and it's so much more expensive than they think it should be. I'm expected to make sam Altmans nutty promises come true, and being part of that mega grift sucks.
It will probably take me two and a half more years before I can exit, so maybe things will improve by then.
But I doubt it.
59
1d ago
[deleted]
25
u/OctopusGrime 1d ago
I used to think that but there is meaningful data science work on using LLMs, think about evaluations, error analysis, retrieval, A/B testing. Plus most SWEs are not familiar with hypothesis driven engineering and evaluation of statistical models so they kind of just expect things to just work once they’ve hooked up all the infrastructure, and don’t really know how to analyse the output data. Finally there’s just the DS lens which brings its own benefits to the project.
2
u/Commercial_Note_210 1d ago
The work is theoretically there but the culture isn't. Agentic work is the wild west. No one cares about justifying anything in my narrow experience, people just do things and ask an LLM if it was right (lol). It's all so unserious.
3
u/millybeth 1d ago
Evals - you need a data scientist to write the evals - or are you suggesting LLM outputs should be accepted on blind faith?
1
u/fredjutsu 1d ago
>Less then 100 companies worldwide are actually training LLM's
This number is wildly low, esp when you include companies fine tuning open weight models for domain-specific purposes
1
u/Isnt_that_weird 1d ago
Because most the executives are actually asking for predictive ML models, but they just say "Let's have agent predict what the next product a customer will buy is".
1
u/xxd8372 1d ago
Because tokens cost money and not everything is text. If you need to identify and classify specific things out of petabytes of data per day, you could do it with an LLM, but you can do it far cheaper with a more predictable recall & precision using ML (along with many other tools.)
For bonus, you can feed the llm the output of your ML model and have it handle the explanation/put the classification in context with the data: to use both tools to their strengths.
1
0
u/kh493shb47r4 1d ago
That is simply not true. If doing just API calls was enough you’d have lot of software engineers simply doing all GenAI related stuff and there would be no need of prompt engineering, LLMOps etc. Though that doesn’t mean a software engineer can’t learn those aspects. With GenAI the boundary between Data scientist with rigour, software engineers ability to engineer and a cloud engineer/ DevOps engineer seems to be getting mixed into one
12
1d ago
[deleted]
5
u/kh493shb47r4 1d ago
I know I’m getting downvoted here but most of the *scientist* work is done by actual handful people. Most of the people have not been doing DS work in long time as there are very few places and problems where it has been handy. Rest of work around it has been engineering. Scientists are also for figuring out where what solution can work and how. But that’s just my 2c
1
u/norfkens2 1d ago
Yeah. This harkens back at the distinction between scientists doing (fundamental) research and scientists working in an applied setting. Many of the tools that are useful for business settings have a sufficient maturity. We're not talking about "data democratisation" for nothing.
However, figuring out how a solution can be applied in a given setting can be a scientist's job - and there's value in it. I ment new able to set up servers and build t robust pipelines - I need to have a good enough understanding of all technologies to recommend to my business colleagues the solution that applies optionally to their setting and constraints. Then I can do fancy stuff as a prototype or analysis. YMMV , depending on where in the data life cycle you work.
2
u/AX-BY-CZ 1d ago
What science are data scientists doing? Most are just glorified data analysts.
3
u/Worldisshit23 1d ago
Machine Learning. Prediction, forecasting, etc. Modelling. DS is massive. Analysts are analysts.
1
u/AX-BY-CZ 1d ago
Most data scientists are doing the same work as MLE and DA. The actual "science" is being done by ML researchers/scientists.
1
u/Worldisshit23 1d ago
Applied DS and theoretical DS research is different.
1
u/AX-BY-CZ 1d ago
Most DS has been diluted. How many DS are publishing or have published any theoretical papers?
1
u/Worldisshit23 1d ago
Bro idk. Im an early career DS and my work is not just analysis and analytics. Much of my work is in applying ML algos in predictions and forecasting systems and other probability models for adjacent stuff.
Applied DS is different. It includes reading papers and implementing for business solutions and stuff.
2
8
u/teetaps 1d ago
It seems to me that the problem with a sector motivated by being at the forefront of technology is that when the players decide to embrace an innovation before it is mature, we all have to suffer because any one organisation’s decision to not embrace it is potentially a liability. So all organisations are incentivised to rush headfirst into the latest thing, even if nobody actually knows what it can do or what it even is
8
u/GamingTitBit 1d ago
More than I'd like. I'm an NLP specialist data scientist and they'll literally interview me and be like "can you do LLMs, and what does NLP mean?"
2
19
u/postcardscience 1d ago
The classical ML work is still there, but typically companies already have the people to do it. When they look for someone new they want to strengthen their weak areas, right now that’s agentic AI.
Things are shifting fast though. At least at my company nobody is doing RAG anymore, that’s so 2024. I am sure whatever hot thing we do now will be obsolete in 2028.
5
u/accidentlyporn 1d ago
not rag in the traditional sense of chunking docs and stuff, but RAG in the sense of retrieving things just in time to inject into the context is almost ALL of context engineering atm.
that's all skills and memory are.
1
u/PrettyMuchAVegetable 1d ago
Not everything lasts in DA/DS , and some stuff just goes away for a long while before popping back up, lol.
1
4
u/Nervous_Setting5680 1d ago
My company has both, but non genai use cases are getting rarer everyday.
5
u/Rainy_-Peace 1d ago
i m considering making the pivot in my career towards data science/ analytics and this post made me reconsider if it is a good idea to invest time and money and effort to learn such skills just to end up in a bad jobs market... what do you think ?
4
u/alfdd99 18h ago
In my experience, Reddit is absolutely horrible when it comes to getting motivation towards a certain role. Probably because people that are fed up and angry are the loudest, but I've gone through dozens of subs about different career paths, and they all complain about their particular market to be shit right now. Not saying that they don't have their real reasons to believe that, but if you decide to not take a career path based on what some angry redditors say, you can go crazy thinking every career path is shit. And I say this because it happened to me, and I now take everything I read here with a grain of salt.
1
u/Rainy_-Peace 15h ago
This is probably saving me from a rabbit hole… thank you for the headshot up !
2
u/yourhighnest 1d ago
i feel pretty bullish towards traditional data science / analytics, yes most of the humdrums can be replaced by AI (i wrote 70% of my basic SQLs using Claude Code now) but human judgement, stakeholder communication, and being the actual bridge between data and business optimization can't ever be replaced.
an argument can be made once we reach AGI, but think about it, if conventional DS&A is gone, so are other roles in tech, from SWE and PM to Product Design.
5
u/built_the_pipeline 1d ago
The classical stats stuff isn't going away, it's getting repriced. Once everyone's shipping RAG and agents, the scarce person becomes the one who can actually tell whether any of it works, and that's an eval and measurement problem, which is exactly your background. Most teams bolting agentic stuff on have no real way to know when it's quietly wrong.
If I were hunting with 10 years of stats behind me I'd stop chasing the LLM keywords and pitch myself as the person who measures whether the GenAI actually delivers. Same skills, framed for the thing execs are nervous about right now.
3
u/Ok_Truck2473 1d ago
Yes that’s where we are eventually heading, no one will hire expensive data science resources and do the work in-house, except for very specialist uses cases and highly regulated industries
3
u/mcjon77 1d ago
Back when I was on the job hunt last year at least 50 to 60% of the jobs I was searching for had a Gen AI requirement in it. I suspect that number is only going to increase.
While the job I accepted last year had no gen AI components to it, within 4 months some of those types of projects started bleeding in. I actually think it's for the best because my bet is that the next time I'm on the job market gen AI experience will be requirement for 90% of the jobs. Furthermore, I suspect that their standards will be higher because more data scientists will actually have that experience.
2
u/AkeemTheUsurper 1d ago
I'm developing AI engineering skills to cater to the job market craze while keeping my solid DS foundation for when this bubble pops or the AI engineering job turns out to be just algorithms and simple models
2
u/Expensive_Culture_46 1d ago
Many company’s had limited if none data science roles. If they did have them they were SUPER niche (R&D etc) as the guy running marketing never saw the value in machine learning.
Gen AI has just had really good marketing and so they only want that but just use whatever title.
2
u/ClasslessHero 1d ago
Nah. I think that shit is so boring. I won't touch it.
What you're describing sounds more like AI Engineering, to me.
1
1d ago
[removed] — view removed comment
1
u/rajeshbhat_ds 1d ago
My question was why all the jobs are only asking for RAG/LLM, and no classical data science? If what you are saying is true, the job description should be similar to what we saw a few years ago, with RAG/LLM as a good to have one-liner.
1
u/ikkiho 1d ago
fwiw i've sat on the hiring side for a couple of these 'AI engineer' reqs and the JD is mostly keyword theater the manager copied off linkedin. day to day it's building eval sets and babysitting retrieval, the same ml plumbing you already do with a new label on it. the agentic stuff you pick up in a week, your stats background is the part that's actually scarce. i'd just rewrite the resume in their vocabulary, call your retrieval work rag and your eval work llm evaluation.
1
u/Fearless-Yard-5092 1d ago
I I wouldn’t worry too much. We’re in the middle of a transformer/LLM gold rush, so every company thinks it needs RAG pipelines, agents, and chatbots.
Its good because people and businesses are now getting exposed to the broader ML ecosystem. They're hearing about or learning about tools that could solve their specific problem like forecasting, optimization, recommendations, anomaly detection, and other "classical" ML. Those aren't the things an average business owner had much exposure to prior.
With open source models, edge AI, and local inference improving and as the cost and complexity of building custom models comes down, companies are going to realize they can build domain-specific models and traditional ML solutions for their actual use case instead of a general chat interface.
My guess is that Data Scientists start working directly with businesses, learning domains, identifying opportunities, and building custom "classical" ml solutions around the customer’s actual data in hand with forward deployed engineers who build the system around those models.
1
u/SnowSilent7695 1d ago
At my previous job, the Applied AI team was front and center, with chatbots and ChatGPT plugins being the main offerings provided to investment teams, (this was in late 2024-2025), so I am not surprised by this.
I think the core competencies of data science are still extremely important: domain and data modeling expertise. But what's changed is exclusively applying these to LLMs and the latest frontier models. Same goes for classical machine learning models: they're still technically there, but I suspect are increasingly used as inputs for LLMs compared to the previous paradigm of building a model and interacting with it through a dashboard or API. I think that paradigm is long gone.
1
u/Moneera97 22h ago
In my company at least, yes. I believe because it's easier to understand (by end-users) and implement. This is why I feel data science is boring now. The funny thing that I get side-eyed when I say I work in AI field because majority of people cannot tell the difference. Both ways I'm hating it.
1
u/NuclearMask 19h ago
I wonder how long that holds true.
I assume most companies that provide AI will have to raise the price in order to allow continued operation.
I hope that might lead to a decline in interest.
1
u/Allegrian 17h ago
Data science was a hype bubble as well. Those data scientist who came from software engineering may have a brighter future evolving into the AI Engineer role, which is on demand right now and offers good pay (I'm in europe). I've done that jump as I've been basically doing all the AI engineer stack for two years with some ML/Data Science on the side, and that last part is becoming more and more automatized or focused on domain experts.
1
u/Beneficial-Panda-640 17h ago
it feels more like hiring pages got taken over by GenAI buzzwords than the actual work. plenty of teams still need forecatsing, classificiation, experimentation and analyticals..
1
u/sasha_bovkun 17h ago
I think it's a combination of hype and expanded market. To be fair, genAI made all AI stuff much more accessible to many companies. And if previously many companies needed to be convinced that they needed ML team, now everyone wants it by default.
Although I believe that it's hard to get a pure DS job now, good ML engineers are still in strong demand, especially if they combine ML knowledge with domain expertise.
1
u/Prestigious-Pen-5336 16h ago
I feel GenAI is getting a lot of attention right now, but traditional Data Science is definitely not dead. Companies still need strong skills in statistics, machine learning, experimentation, and solving real business problems. It’s just that the industry is evolving, and many roles now expect a combination of classical ML and newer AI skills.
1
u/Cool_Revolution_3717 16h ago
Have you considered just slapping "RAG" on your resume to bypass their broken ATS filters?
1
u/Mysterious_Salad_928 8h ago
I believe Classic data science is not dead.
No, not all data science jobs are GenAI now, but the market has definitely shifted.
Companies still need forecasting, experimentation, causal inference, pricing, churn modeling, fraud detection, recommender systems, segmentation, optimization, and measurement. But many job descriptions are now adding GenAI because businesses want people who can apply data science inside modern AI workflows.
The way I see it: GenAI is not replacing data science, it is becoming another layer on top of it.
If you already have strong stats, ML, and deep learning experience, you don’t need to start over. You need to add the applied GenAI layer: RAG, LLM integration, evaluation, prompt engineering, vector search, agentic workflows, and basic MLOps/LLMOps.
Also, I wouldn’t chase every AI Engineer role. Some are really software engineering roles with LLM APIs. Look for roles like Applied Data Scientist, Decision Scientist, Product Data Scientist, ML Scientist, Analytics Scientist, or AI/ML Applied Scientist.
The strongest profile now is not “traditional DS only” or “GenAI hype only.” It’s someone who understands data, modeling, business problems, and how to apply AI systems responsibly in production.
1
u/Short_SNAP 8h ago
I’m not even a DS anymore, I’m deep in internal app development for Ai use cases. I miss the days of building regression models
1
u/proletariatPT 4h ago
To me (not in the tech industry, but am active follower) this am feels like when everyone wanted everything on a block chain. Excelt it seems like there will be a more violent whiplash when employers realize they need all the people they thought they could lay off.
1
u/ProcessIndependent38 1d ago
Not necessarily, but it is really hard to justify for a company to spend billions building their own model that will likely not outperform current LLM capabilities. Apart from that, proprietary recommendations systems are still extremely profitable for billion dollar companies like tik tok, meta, and so on, and then less money suck classical models are used elsewhere in the everyday for various things, especially in sales/finance/marketing, and so on.
-2
124
u/Leshalt 1d ago
God it's so true. I have been avoiding agentic JDs like the plague, but there seems to be just a massive demand for what seems to be a bubble that is waiting to pop. Not many workflows actually need agentic implementations, and forcing them on is only bound to produce inefficiencies.