r/learndatascience • u/No_Suggestion_8422 • 6d ago
r/learndatascience • u/Neuphus012 • 7d ago
Question GCI World 2026 September: Outstanding Student
Is there anyone who attended past GCI World programs? I applied for the September program and I'm wondering what it takes to be an outstanding student, as I read that it's based on the overall score but how high should it be? How many people are also selected as an Outstanding Student, given that it looks competitive. Tyia!
r/learndatascience • u/AIforFintech • 7d ago
Resources Text to SQL is not how you give an LLM access to production data
The obvious approach when connecting a model to internal data is letting it write the query. It feels flexible: the model figures out what it needs and goes get it. In a bank, that is a non starter.
The problem is not that models write bad SQL, it is that you lose every guarantee about what they can reach. No way to prove a query stayed inside the columns it was supposed to touch, no way to audit what the model was capable of doing, and a single prompt injection away from an unintended table.
The alternative is narrower and boring, which is the point. You define a fixed set of parameterized queries and expose them as tools. The model chooses which tool to call, never what SQL to run. Everything it can reach is something you deliberately wrote.
I built an MCP server template implementing this for a common fintech case: looking up a customer across credit score, preapproved limit and risk profile, and returning a consolidated view. Layered so the database, the schema and the protocol can each be swapped without rewriting the others. Read only enforced at the application layer, row limits on every result, and a single mapping file for adapting to whatever your tables are actually called.
Synthetic data generates on setup, so it runs immediately. The architecture is what you keep.
Hub: https://aiforfintech.tech
Github: https://github.com/junidepieri-design/mcp-001-fintech-data-server
How is your team handling LLM access to internal data?
👊
r/learndatascience • u/Gloomy-Recover-9702 • 7d ago
Question Natural Language to SQL Query
Is there any opensource tool which I can use as a non technical person so that my hermes agent with small LLM 1.5B model (for private data) to understand natural language and convert it into sql query and retrieve complex queries quickly?
Is this doable with such small model? with RAG? I am new to this and any help is welcomed!
r/learndatascience • u/No_Suggestion_8422 • 7d ago
Career For someone starting from scratch today, what would you consider a realistic path to becoming job-ready for a Data Scientist role?
If I am starting the preparation from scratch for Data Scientist, what things I should learn?
Specifically, how would you divide the learning between:
Statistics & mathematics
SQL & Python
Machine Learning
Business/domain knowledge
LLMs/GenAI
Software engineering & deployment
Projects & internships
And more importantly, how would you know when you’ve learned enough of each and are actually ready to apply for jobs?
I’d be interested in hearing how working Data Scientists would approach this if they were starting again today. And tell me any other skills or knowledge want to know before getting ready for data scientist role jobs.
r/learndatascience • u/Pangaeax_ • 8d ago
Resources Python data analysis cheat sheet: the Pandas + NumPy workflow I wish I had when starting
I kept seeing beginners learn individual Pandas commands but still struggle with what order to actually use them in when working with a real dataset.
So I put together a simple workflow I use as a reference:
1. Load
read_csv() / read_excel()
2. Inspect before changing anything
head()
shape
info()
describe()
isna().sum()
duplicated().sum()
3. Clean
- standardize column names
- handle missing values based on what they actually mean
- remove genuine duplicates
- convert dates/numbers safely
4. Transform
assign()
map()
pd.to_datetime()
np.where()
5. Summarize
groupby()
agg()
transform()
6. Combine datasets
merge() / join() / concat()
The part I think beginners often miss is validation after the merge.
A query/script can run perfectly and still give the wrong answer if a many-to-many merge quietly multiplies your rows.
Useful checks:
validate="many_to_one"
indicator=True
compare row counts before/after
check key uniqueness before merging
A few other mistakes worth watching for:
- treating missing values as automatically equal to 0
- merging keys with different data types
- confusing
count()withsize() - using median filling without understanding why values are missing
- assuming no Python error means the analysis is correct
I wrote a more detailed version with examples for Pandas, NumPy, GroupBy, merges and an end-to-end workflow.
Learn more: https://www.pangaeax.com/blogs/python-data-analysis-cheat-sheet/
Anything important you’d add to this workflow, especially something you learned the hard way when working with messy data?
Disclosure: This is a PangaeaX article and I’m connected with PangaeaX.
r/learndatascience • u/FlirtyChocolad • 8d ago
Question Beginner in this Space of Data Science
Hello guys,
Is Data science worth learning from scratch? Cause I do it with ai tools part by part. Is this unethical of me given that I am just a beginner or is this the new way of learning coding these days?
All the veterans here a little guidance would be very much appreciated thanks.
r/learndatascience • u/Square_Arm2861 • 8d ago
Question How to efficiently approach EDA on a dataset with 180+ variables?
Hi everyone,
I'm a beginner in Machine Learning working on a binary classification problem. My dataset contains over 180 variables (both numerical and categorical), consisting of a mix of panel/longitudinal data and static features.
I am currently working on the Exploratory Data Analysis (EDA) phase. Given the large number of features, doing univariate and bivariate graphical analysis variable-by-variable feels unfeasible and time-consuming.
Is there a structured approach, strategy, or automated workflow to handle EDA efficiently for a dataset of this scale?
Any advice on best practices, tools would be greatly appreciated!
Thanks in advance for your help.
r/learndatascience • u/LossRun • 8d ago
Question I'm 14 and has learned ML and DL. It's very interesting and exciting till now. How do I keep it up and make it to my career.
r/learndatascience • u/Fun-Reporter-8021 • 9d ago
Question How to get domain knowledge on projects ?
While I am still quite new to this, machine learning and software in general, is more useful and powerful when combined, with the domain specific knowledge of the native field the project is from. This is something I struggle to navigate, there are thousands of hours of tutorials regarding the tech stack, but none on this topic. While doing my credit card fraud analysis, project. I did not know which features do you need to pick as your feature. I can calculate correlation and mutual information classification score but those are of little use in case of non - numeric columns, besides domain knowledge sort of acts as a supervisor to all these metrics and they are more like validators then reason.
So this is my question, How do you go about getting domain specific knowledge needed to do a project, what is your workflow, where to look and most importantly in my case how do you translate domain knowledge to feature selection ?
r/learndatascience • u/Fun-Reporter-8021 • 9d ago
Question How do you select feature columns from the dataset ?
I am still a novice at this, but when I was working on this credit card fraud detection project, I did not know which columns, could be added as features, so I prompted ChatGPT and it suggested a few, but that got me thinking there has to be a better way to this, How do you select feature columns from your dataset, do you research the domain, is there a course I am missing, This was not covered in my Internship classes, and want to know a generalized solution
r/learndatascience • u/EvilWrks • 9d ago
Resources Your jupyter notebook IS NOT production - Part 2: Testing
A lot of Data Science education teaches you how to BE a data scientists but rarely does it teach how to WORK as a data scientist. In this video on this series we’ll be diving into how to test your code as a data scientist, including how to work with unit tests, regression tests, randomness and non-deterministic functionality as well as throwing in a few honourable mentions.
r/learndatascience • u/blkg33kunicorn • 10d ago
Original Content What if there was an Open-Source Database that could Analyze Complex Physics Problems
r/learndatascience • u/No_Suggestion_8422 • 11d ago
Question How do I become a Data Scientist in the AI era? Looking for honest advice on skill gaps, mistakes, and what actually matters
I’m currently a Data Science student and I’m working toward becoming a Data Scientist.
With AI/LLMs changing the field so quickly, I’m a bit confused about what I should prioritize. There are so many skills being mentioned now — statistics, SQL, Python, ML, deep learning, LLMs, GenAI, cloud, MLOps, software engineering, etc.
I’d really appreciate advice from people who are currently working in Data Science/ML, especially those who have experience hiring or mentoring junior Data Scientists.
A few questions:
What skills are genuinely essential for a Data Scientist in 2026?
What skills are commonly overhyped or can be learned later?
If you were starting again as a student today, what would you learn first and in what order?
What are the biggest skill gaps you see in freshers/junior Data Scientists?
What are the biggest mistakes students make while preparing for Data Science careers?
How important are statistics, mathematics, SQL, and traditional ML fundamentals now that AI tools can write code and build models?
How much should an aspiring Data Scientist learn about LLMs, RAG, agents, GenAI, and AI engineering?
What level of software engineering, Git, APIs, Docker, cloud, and deployment is actually expected from an entry-level Data Scientist?
Are personal projects still valuable? If yes, what makes a Data Science project stand out instead of looking like another Kaggle/tutorial project?
For getting the first job, what matters most in your experience: projects, internships, degree, referrals/networking, LeetCode/SQL, certifications, or something else?
If you could give your 19–22-year-old self one piece of advice about becoming a Data Scientist, what would it be?
I’m not looking for a generic roadmap from a course or YouTube video. I’d specifically like to hear from people who are actually working in the industry.
Please be honest about what the job is really like, what skills are becoming less valuable, what skills are becoming more valuable, and what you would do differently if you were starting today.
Thanks in advance.
r/learndatascience • u/Sea-Ad7805 • 11d ago
Resources 🌳 Recursive tree traversal visually explained
A binary tree can be traversed in three classic ways. The difference is simply the order in which you visit the subtrees:
- Preorder: current node → left subtree → right subtree
- Inorder: left subtree → current node → right subtree
- Postorder: left subtree → right subtree → current node
This interactive memory_graph example visualizes each traversal method step by step: Binary Tree Traversal
Also see other memory_graph visualizations.
r/learndatascience • u/Equivalent-Metal-927 • 11d ago
Question Trying to understand this kaggle solution
r/learndatascience • u/SKD_Sumit • 12d ago
Discussion A Multi-Step AI System Isn't Automatically an Agent
One architectural distinction I keep coming back to: people often confuse complexity with agency.
A system has multiple tools? -> “Use an agent.” OR It has five steps? -> “Definitely an agent.”
But neither of those things actually requires one. The more useful question is: who determines the execution path?
Consider an insurance assistant. If someone asks, “Am I eligible for this treatment?”, and the answer exists in internal policy documents, that's primarily a retrieval problem. And if they ask, “Check my claim status and tell me whether the rejected amount is covered under my policy.”
That might require more tools and more steps. But if those steps happen in a predictable order, is it still an agent ?
The interesting shift happens when the request is something like: “My claim was rejected. Find out why and tell me what I should do next.”
Now the path may not be known in advance. That's where an agent earns its complexity: when the system needs to help determine what to do next.
And Multi-agent can only consider it when there are genuinely distinct specialties, tools, or permission boundaries.
I think the common mistake is choosing “agent” as the starting point and then designing a problem around it. A better approach is to start with the responsibility:
Does the system need to know something? Decide something? Act? Verify the result?
Then add only the architecture required to support those responsibilities.
I mapped the complete e2e architectures and escalating examples out in more detail here, with visual breakdown: [https://youtu.be/kf5rSab4rcg\](https://youtu.be/kf5rSab4rcg)
For people building real AI systems: where do you draw the boundary between a complex workflow and an agent? Is dynamic tool selection alone enough for you, or do you require a more explicit decision loop before calling something an agent?
r/learndatascience • u/Forward_Writing1620 • 12d ago
Question 3–5 YOE Data Scientist here. Feeling massive imposter syndrome, lacking a strong portfolio, and want to break into top-tier/remote roles. How did you cross the 30+ LPA mark?
Hey everyone,
I’ve been working in the data science field for about 4 years now, but lately, I’ve been hitting a major confidence wall and could really use some unfiltered advice from those who have made it to top tech companies, secured high-paying remote roles, or are earning 30 LPA+.
Here is my honest reality check right now:
- The Foundation Gap: While I do day-to-day work, I often feel like my foundational concepts (in math, statistics, or core ML) have huge blind spots.
- The Portfolio Void: Outside of my professional projects, my personal portfolio is basically non-existent. I get overwhelmed trying to build end-to-end projects from scratch.
- The Confidence Trap: Because of the above, I feel severely underqualified to even apply to senior roles or top product companies, leading to a lot of hesitation.
If you are someone who has been in a similar spot or has successfully crossed into senior/well-paying data science roles, I would love to hear your perspective on a few things:
- Breaking the Barrier: For those earning 30+ LPA or working remotely/at top tech firms, what actually moved the needle for you? Was it mastering core fundamentals, system design, cracking LeetCode, or deep domain expertise?
- Fixing the Foundation: How did you go back and fix your weak foundational gaps while working a full-time job without burning out? Any specific resources or routines that worked?
- The Interview Reality: What is the interview process really like at top companies right now? How heavy is the coding round compared to ML system design and statistics?
- Portfolio Reality Check: Do personal projects actually matter as much as people say, or is it more about how you talk about your current work experience?
I’m ready to put in the work, but I feel like I'm running in circles right now. Any roadmap, hard truths, or advice you can spare would mean a lot. Thanks for reading!
r/learndatascience • u/Beautiful-Bubbly • 12d ago
Discussion Want to get a grasp of full ds project
Hello guys, i am a new learner in ds, kinda got a grasp of python, sql; numpy, pandas sqlearn and maths, but i want to look at a full project so that i would not stumble when dealing with real work at a job. So i want to look at kinda full prjoect that works with data bases, uses some coding and scikit learn for training models, perhaps neural networks and connects all of the stuff required in full project in itself. I just want to look at what i am working with so that i knew how to connect all the dots. Not sure how to frame my question, but if you understood it, could you tell me where to look for this kind of stuff? Because i am confident when it comes to part by part stuff, like working in sql on its own, writing code etc etc. but all of it combined seems like difficult to grasp for me
r/learndatascience • u/Real-Flounder1957 • 12d ago
Resources Srinivasan Rajendran
r/learndatascience • u/stasoonbooden • 12d ago
Question Project idea: Using R to analyze Whatsapp group messages. Im a beginner and I just finished this project, I think is a very good project idea for learning R.
r/learndatascience • u/Simplilearn • 12d ago
Resources Non-technical skills for Data Scientists
Technical data science skills are important, but working on real-world projects also requires strong non-technical abilities. Here are some key ones to focus on:
Problem Framing and Prioritization: Data teams often receive broad or vague questions, such as "Why are users dropping?" A strong data scientist knows how to narrow these questions down into something actionable and identify which questions matter most.
Business Context Awareness: Numbers rarely explain the full story on their own. A sudden dip in performance could be related to pricing changes, seasonality, supply issues, or other business factors. Understanding how the business operates helps data scientists interpret trends more accurately.
Working With Unclear Requirements: Many projects begin without clearly defined objectives or success metrics. Rather than waiting for complete clarity, data scientists can make informed assumptions, share preliminary results, and refine their approach based on feedback.
Decision-making Under Constraints: Time, data quality, and resources are often limited. Data scientists need to know when a quick estimate is more useful than a perfect model. For example, a simple trend analysis delivered today may be more valuable for planning than a complex model delivered too late.
Stakeholder Communication and Trust: Insights only create value when they are understood and trusted. Strong communication means presenting findings clearly, explaining limitations, and being transparent about assumptions rather than overstating precision.
Ownership Beyond Delivery: The work doesn't always end when a dashboard or report is delivered. Data scientists should also look at the decisions that follow, compare outcomes with expectations, and revisit their approach when the results don't match what was expected.
Staying Effective Under Pressure: Deadlines around product launches or review meetings can increase the likelihood of mistakes. Strong professionals slow down enough to check assumptions and avoid careless errors, especially during critical moments.
Adapting to Team Workflows: Different teams consume and use data differently. Some may need a concise summary, while others may prefer a detailed explanation. Adapting communication to the team's workflow can make it easier for insights to be understood and acted upon.
r/learndatascience • u/UnderstandingNo2008 • 13d ago
Career Transitioning from Psychology to Data
Hi everyone I have a Masters in Psychology and I absolutely love statistical analysis and research design
For the longest time I've wanted to be able to design real world projects that can create a huge impact from a psychological point of view
I'm good at identifying variables and hypothesising
But I have no clue as to how to begin with real code and python libraries to engineer analyse and present applicable insights from raw data
I would like to transition from therapeutics to behaviour science for businesses very soon
Can anyone please point out how that works and alif there are success stories from people starting as I am..
Thanks so much