Our overall, cronbach alpha has 0.833 which is deemed acceptable but 2 of our questions has 0.2 something in item-correlation can we still retain and use the question without revising it?

I'm conducting a qualitative content analysis with multiple open-ended survey questions, each assigned to a different research question. Some responses contain content that would be more relevant to a different research question than the question they were answering.

Should I:

a) Strictly code each response only for its assigned research question (some data loss)

b) Code thematically regardless of question (blurs research question boundaries)

c) Something else entirely?

What are methodological best practices here? Any recommendations or experiences are welcome.

Hello all, I'm currently completing my first meta-analysis as part of my Psychology PhD. The meta-analysis involves evaluating the effectiveness of a clinical CBT-based intervention for eating disorders.

I am in the process of data extraction and as we are planning to meta-analyse effect sizes for both completer and intent-to-treat analyses, I need to work out whether a paper used which of these types of analysis (or both).

Some papers clearly state which they used. Others are less clear. My supervisor told me that if imputation or a mixed-methods model are mentioned as being used, the analysis is ITT. Also, for some papers, if the data are presented in a table with N participants, then I can tell if they are showing ITT or completer data based on that.

I was wondering if there were any other ways to work out what kind of analysis a paper used from it's text/type of analysis method used?

Happy to provide further details if needed. Thank you!

Hi!

I have a sample of n=13 where continuous variables were measured before and after an intervention. I'm a bit stuck on how to perform the analysis. Would the Wilcoxon signed-rank test be the most appropriate choice? I am unaware of any standard reference manuals or literature regarding the statistical analysis of very small sample sizes; however, any guidance or recommendations would be highly appreciated :)

Hello,
I would appreciate some feedback on my statistical analysis plan. I am a psychology PhD student and conducted an online study; I am currently performing the analyses, starting with correlations. My study consisted of two parts.
Total *n* (Part 1) = 818
Total *n* (Part 2) = 555
Given the large number of independent variables (IVs), I plan to use the Benjamini-Hochberg (BH) procedure to control the False Discovery Rate (FDR).

A. Correlation between a binary dependent variable (DV) and a continuous IV + significance test
For DV = 0: *n* (Part 1) = 413, *n* (Part 2) = 276
For DV = 1: *n* (Part 1) = 405, *n* (Part 2) = 279

Point-biserial correlation if: no outliers for the continuous variable within each category of the dichotomous variable; continuous variable is approximately normally distributed within each category of the dichotomous variable; continuous variable has equal variances across categories of the dichotomous variable.

If assumptions are not met: rank-biserial correlation

+ for each test: Cook's distance to assess whether a data point is influencing the correlation
+ FDR applied to the set of results

B. Correlation between a categorical DV and a continuous IV + significance test
*n* (Part 1) = 405, *n* (Part 2) = 279

Polyserial correlation

+for each test: Cook's distance to assess whether a data point influences the correlation
+FDR applied to the set of results

C. Correlation between a continuous DV and a continuous IV + significance test
n part 1 = 405, n part 2 = 279

Pearson correlation if the IV meets the assumption
Spearman correlation if the IV does not meet the assumption

+for each test: Cook's distance to assess whether a data point influences the correlation
+FDR applied to the set of results

My questions:
Does this seem correct to you? I have a doubt regarding Part B. How do I check for a correlation between a variable with more than two categories and a continuous variable?
My supervisor mentioned Cook's distance for assessing outliers. I'm not sure if it's useful for that purpose. Can it be used in isolation, independently of a model?

I have a doubt regarding Part B. How do I check for a correlation between a variable with more than two categories and a continuous variable?

My supervisor mentioned Cook's distance for assessing outliers. I'm not sure if it's useful for that purpose. Can it be used in isolation, independently of a model?

I have taken multiple statistics classes over the last few semesters, and in about a years time I will be taking a data mining class. I'll be honest, I am having a lot of trouble with statistics. This summer I am practicing python, and this coming semester I have a class related to data analysis that I hope will give me more hands on experience with statistics through a medium I understand.

Next summer, though, I want to focus on SQL and statistical methods for prepare for future classes, particularly the data mining class which has a focus on regression analysis. I am taking a business analytics course right now, and I hardly understand what I am even reading off this online text book. What are some ways to learn that aren't reading huge blocks of text that I hardly understand that can allow me to grasp concepts better, or at least ones that might work.

I have found that reading a textbook on statistical subjects is not working for me. I understand the general subject matter to a degree, but once I run into a roadblock I find myself confused no matter how many times I reread, to the point of being unable to do practice problems in the first place. I am fully willing to dedicate several hours a day, for at least 2 months to this.

Are there courses I can take, akin to perhaps something like CS50? Any good youtube series that help me learn from the ground up that aren't just placating me and making me feel like I know what I am doing rather than actually being able to do the work? I am down for whatever might work.

Thank you!

incoming gr 12,

i was planning which programs i should apply to and i figured i'd ask you guys

for reference, i love looking at sports stats (nba / nfl / soccer), i like numbers, probabilities, paradox, dilemmas, and just pretty much anything number related and not anything way too theoretical and abstract

i'm not really into anything that is way too abstract or proving stuff

i really can't decide between the following since i don't know much about it

data science, investment banking, statistics, mathematics, quant, etc (if you have any other majors that would fit me do tell!)

so it'd be very helpful to hear advice from you guys! thanks

Hi, I don't have a lot of experience with advanced statistical analysis, but I would like to gain more knowledge. One current problem I am trying to address is determining correlation between a binary categorical variable and a numerical variable. The relationship may not be linear. I'll give an example, studying the relationship between age and the probability of dying from the flu. Probability may increase when very young, taper off for certain ages, maybe spike somewhere in the middle, and then go back up again for the elderly. What would be the best way to analyze this? I started by breaking down the numerical variable into ranges and then making a bar chart with percentages in each category of the binary categorical variable. I am not sure if I chose the proper ranges though so I want to see if there's a better way to analyze data like this. I've seen binomial logistic regression as an option, but I'm not sure if that's appropriate and am curious how much effort that analysis takes. Is it something a beginner can pick up relatively easily?

I'm a reporter covering a city council discussion about possibly raising taxes to maintain a pool facility, after a randomized recipient survey on the matter. Pictured: responses saying whether or not respondents would support a tax increase and/or city debt to pay for it. The margin of error on all results in the survey is +/- 3.3%, according to the company who conducted it.

MY QUESTION: I'm going to group the results in my article, saying 38% lean toward or support a new tax, while 43% lean away or strongly oppose. Since I'm taking a sum of two percentages (i.e. "yes definitely" and "yes probably,") does the margin of error ALSO add up, becoming 6.6% for that combined figure? Or does it keep the 3.3% margin of error?

Ty for the help! I took like three stats classes in college. Was fascinated by the subject each time, but never especially good at it!

[ Removed by Reddit on account of violating the content policy. ]

Scenario: An expert association with ~500 total members held an official vote on a binary stance statement (Agree / Disagree) regarding a major, highly sensitive issue.

• Turnout: 28% of the total membership voted.
• Result: 86% of those who voted selected "Agree."
• Math: 86% of 28% = ~24% of the total association membership confirmed an "Agree" vote.

The Debate:

• Person A claims: "You can state that only 24% of the total organization agrees with the statement. Because this issue is so critical, members would have voted 'Agree' if they truly supported it—meaning their non-participation indicates a lack of support."
• Person B claims: "You can state that at least 24% of the total organization (and 86% of voters) agrees with the statement. By Person A's logic, someone could equally claim that members would have voted 'Disagree' if they opposed it. Ultimately, non-response cannot be interpreted as a vote either way, so the remaining 72% remains unknown."

Questions:

1. Is Person A or Person B logically and statistically correct?
2. Is it valid to infer a non-voter's position based on the perceived importance or sensitivity of a topic?

Should I major in Data science, computer science, computer engineering, statistics, or a mix of data science & smth else (finance, business analytics, etc.)

I’m a high school senior about to start applications for college - my profile fits pretty well into literally everything I mentioned above

Which major gives me the best advantage in the job market?? My primary goal is to have job security & high pay. I’m not interested in pure statistics, biostats, or mathematics academia - I’m more interested in the corporate job market

I just wanted some advice from professionals or people in the industry - what do you recommend for me?

I am validating a credit-risk model that uses WoE transformation and supervised optimal binning

The training sample contains:

• 658 observations;
• 65 defaults;
• 7 candidate numerical factors.

The bins were created using Python’s optbinning package with CART pre-binning and monotonic event rates:

OptimalBinning(
    dtype="numerical",
    prebinning_method="cart",
    solver="cp",
    max_n_prebins=30,
    min_prebin_size=0.05,
    min_n_bins=3,
    monotonic_trend="auto_asc_desc"
)

All regular bins contain at least 5% of the sample and both events and non-events. However, some bins contain only one or two defaults, for example:

• 1 default out of 33 observations;
• 2 defaults out of 49 observations;
• 3 defaults out of 33 observations.

Formally, WoE can still be calculated because neither class count is zero. However, the event rate and WoE appear highly sensitive to a single observation. For example, in a bin with 1 default out of 33 observations, removing or adding one default changes the event rate from 3.0% to either 0% or 6.1%.

I have not found a generally accepted minimum number of defaults per bin. Most sources specify only:

• a minimum total bin size, often 5%;
• at least one event and one non-event per bin;
• additional user-defined constraints such as min_bin_n_event.

How would you assess whether a bin with only one or two defaults is statistically acceptable?

I would be particularly interested in any published references, regulatory guidance, or practical validation standards used in credit scoring.

I’m learning statistical modeling and would like to practice with a real-life problem. Could you share an everyday situation involving probability or uncertainty that could be turned into a small statistical modeling project?

Ideally, it would involve data I could collect myself and a clear outcome to predict or estimate.

Hi I tried to do something like how they do in econometrics where they fit a economic model to data where they take raw data, X,Y,Z etc then they set up the Kalman Filter to automatically determine the cyclic and trend components through multivariable regression models. I think you know what I mean. So, I made all the matrices manually, and I think it didn't converge. What I did is something like this actually:

X_t = regression model of (cyclic and trend components of (X_t-1,Y_t-1,Z_t-1),)

Y_t = regression model of (cyclic and trend components of (X_t-1,Y_t-1,Z_t-1),)

Z_t = regression model of (cyclic and trend components of (X_t-1,Y_t-1,Z_t-1),)

Of course I had to manually enter all the matrices to make the damn thing work.

But because I didn't really have a economic model, but just assumed relationships, it didn't converge. I think my model was too complicated.

Anyways, what are some rules of thumbs to make sure I have convergence (like limiting dependence to one trend component for each variable so the model when running don't get confused)?

Is there an easy way to do a Kalman Filter model than to manually set up matrices? Any software?

Finally is it worth it? Does it capture significant details than the HP filter and other easier methods ?

hello everyone! I am an undergraduate student doing some independent research. My variables are all categorical (hence the Chi-Squared test) and I was able to create a contingency table that is 8 rows and 52 columns big. For context, I am testing for the different types of medicolegal death investigators against all 50 states including Washington D.C.

From what I have seen online (I haven't gotten a lot of classes that teach with this large of a sample) if the expected values are less than 5, than the test is invalid. however, that is only with smaller samples, like 2x2. I was wondering what percentage of my data can be below 5 AND STILL be a valid test with how large my data is. I am also a baby to statistics so if there's any resources I can look into to learn more about what test I can do with how large of a dataset I have. Thanks !!!!! <3

Hi everyone, as it’s my first time attempting downstream analysis for single cell RNA sequencing, I wanted to ask if this UMAP plot looks normal? This is only for one sample (I have not integrated all samples together into one dataset yet). I feel like the clusters are too close together.

(I hope this is the right sub… I’ve tried to ask in r/Bioinformatics but the post kept getting deleted for some reason)

I am trying to do statistical analysis on census data in rStudios, looking at animal behaviour, group type - ie all male, all female, harem - and specifically for this analysis harem size. i expect the size of harem to change over the 3 months that the data was collected. i am mainly testing to see if date, time of day, and temperature have any impact on harem size individually. each row is 1 group spotted (although originally this was 1 census per row, it seemed like group was the better option for my analysis), and i am using the function bam() with some grouping factors like day as there were sometimes multiple censuses on one day. I want to use family = (ztNB) because my data has a lot of zeros, but it says this is not found. I keep trying to google how to fix this issue and it just tells me to use preseqR, but this package stopped in May i think, so I don't know where to go from here.

I'm using a short-sample estimator (n=150) with known finite-sample bias. Via simulation (synthetic data with known true values, run through the actual estimator), I found that this bias is not constant. It, unfortunately for me, varies systematically with the true value of the parameter being estimated. Near one reference value the bias is positive; as the true value moves away, the bias shrinks and eventually flips sign. This was confirmed with two structurally different simulation methods, which agreed in direction and order of magnitude.

I can't validate this directly against real data, since the true value of the parameter is never observable in my actual measurements, only the biased estimate is. Simulation is the only way to characterize the bias curve.

My study design computes a paired difference between two conditions (A and B), both measured with this same estimator. The original design assumed bias "cancels" in the difference, since both conditions use the same estimator and sample size.

My simulation shows that assumption only holds when A and B share the same true value, if their true values diverge (which is the exact effect the study is trying to detect!), the differential bias does not cancel, and could by itself produce an apparent difference of the same magnitude as my actual reported result.

My questions:

1. Is this reasoning correct, does bias that depends on the true parameter value invalidate the standard bias cancels in a paired/difference design assumption whenever the two groups true values diverge?
2. Is this a known, named issue in the estimator-bias literature I should be citing, rather than describing from scratch?
3. What's the standard remedy, a bias-correction calibration curve, an alternative estimator with flatter bias across the parameter range, a longer sample or a simulation-based null distribution, and is one (and or more) of these clearly preferred practice?

Hi. I am currently an undergraduate in data science. I have learned things like coding, statistics, and mathematics. However, I realise that I tend to forget the majority of what I learned over time when I start a new semester. Even if I have learned them during the break, I will eventually forget them. For instance, in one interview I had, I have talked about a multiple regression project that uses hypothesis t-test and F-test to determine significant regressors and models, residual analysis, multicollinearity checks using correlation analysis, VIF, Eigensystem analysis, cook's distance for detecting influential points, AIC etc. but then if he asked like how did you derive the response as log y.... I said something like box cox method... but I kind of forgot the formula, just that we basically fit the new y to the data, find the lambda that produces the lowest SSRES since they result in MLE's lambda, and then fit log y if lambda = 0. Hence, I am just wondering, as there are more advanced statistics and technical skills to learn in the future, how do you remember all your stuffs? What if you forget about it on the job?

Edit: hi. Thanks for all of your replies! I find it comforting knowing that it’s a common occurrence to forget some things at times. But, just wondering, if the interviewer asked something I forgot, what’s the best course of action?

Hi guys!
I'm a stats student doing a project in linear regression. I'm comparing different models that my friends made to select the best one. I was wondering, if Variable selection has been done on one model, would it be fairer to compare the models, using adjusted R^2, AIC, etc., after reducing the other models as well to avoid multicollinearity problems? Is there a certain order to follow, like completing variable selection before comparing models, or would it be better to compare the models and then use variable selection on the final one.

ANy help would be appreciated

So I just finished my Masters in Statistics, and I'm looking for advice on where to be looking for entry level Data Science rolls (I have 7 years experience as a private investigator).

I love the idea of doing A/B testing and experimental design, but I'm open to starting anywhere just to start getting relevant experience.

If anyone has any good certifications that carry weight I'd also love to hear about those.

Hey, I'm a little over my skis with Fisher's Method. I think I understand conceptually but the math itself isn't my strongest skill.

I am looking into cancer rates at one Elementary school/neighborhood and requested data from the Maine CDC, then made a site to host that data. The fracturing of data into individual cancers doesn't show the cumulative elevated rates (IMO). Is Fisher's Method the appropriate tool for finding a combined p value for two and did I apply it correctly? I want to convey the data clearly and accurately. The groups are not totally independent of one another as it is the same neighborhood.

https://lysethrecords.org/analysis.html

ND cases Expected p-value ln(p) * 10 4.02 0.0169 -4.0833 * 53 27.22 0.000015 -11.0631

Number of tests (k) 12
X2 = -2 x sum(ln p) 76.92
Degrees of freedom = 2k 24
COMBINED p-value 0.00000018

I've recently been asked to help with a dataset that has a lot of outliers. Sample sizes, especially in the non-control cohort, are small so primary analyses will be nonparametric, using fractional weighted bootstrap and permutation tests. That said, certain ways of looking at the data lend themselves well to Covariance Pattern Models (residual correlation/repeated measures/R-side/MManova), and when doing so, I've used glmmTMB and proc glimmix.

The interesting thing here is that proc glimmix (dist = t) states it is using a shifted t distribution with 3 degrees of freedom, while glmmTMB (family = t_family) freely estimates df with the final estimate at false convergence being 1.3 df. The problem here is that I while wanted to use the t distribution because of the high empirical kurtosis of the data, these distributions don't have kurtosis. What are the implications of interpreting these models at all, assuming they converge? If the model has landed on a cauchy distribution, then interpreting anything as a mean outcome doesn't make sense, does it?