CHEMOMETRICS Flashcards
When do we use permutation tests or bootstrapping
when the observed data is sampled from an unknown or mixed
distribution
low sample sizes
Where outliers are a problem?
Too complex to estimate
the distribution?
Note this is an alternative to non parametric approaches
How do permutation tests work ? under what basis
Assume that if A and B are the same then labels don’t matter
so if testing to see if groups A and B are different
Steps:
1) calculate observed test (for example t test often non parametric- can be anything, ANOVA, quadratic etc) - called to
2) place all in a single group
3) - randomly assign to groups of equal size
4) calculate new test stat
5) repeat - for every single possible random placement into groups
6) arrand all the tests stats in ascending order - this is an empirical dist based on the data
7) if t0 falls outside the middle 95% of the empirical distribution then reject null hypo
What is an exact test vs approximate test in permutations?
exact does all the possible combos whereas approximate samples from all and samples some
What is Bootstrapping
Generates an emprical distribution but based off replacing the members of the original sample with other random members of the original sample (sampling with replacement) - basically just make a bunch of data sets with the same # of samples using those original values and that’s the equivalent of running the experiment a bunch of times - this way we can see where the data really lies instead of having just one set
(again can do with any stat)
What is Jackknifing
It’s a mean to estimate variance by doing subsampling (randomly leaving out samples from the set
What is K fold cross validation
used to validate a predictive model - splits data into K subsets each held out in turn as a validation set to test
What is a time series?
longitudinal data sets - over time - they plot the data (what happened) but also try to predict what happens next (forecast)
What are the steps in time series analysis
1) visualize data
2)Smooth /clean -
3)decomposition (eg if seasonally such as monthly or quarterly - can be decomposed into trend component (change in level over time)
4) show irregular components (not part of trend
What are trends people see in time series
They see additive trend (increase over time)
Additive seasonal (see it go up and down with seasons - almost sinusoidal)
and multiplicative trend (with seasonal gets larger/wider)
How are things smoothed in timem series
movign average - average points next to you - k = how many points
Exponential forecasting models
single - a series with constant level and irregular component (no trend or seasonal)
Double (holt) - exponential- series with a level and a trend
Triple (Holt Winters) exponential- series with level, trend and seasonal
Types of Error
I - alpha rejection of true null hypothesis (false positive)
II - beta - non rejection of false neative
What is LOD
lowest amount of analyte in sample that can be detected WITHIN a specific confidence level
is LOD agreed upon?
no - typically s/n relationship
Draw curves for signal to noise and blank and what shades represent what
Used for LOD determination - want stdev of blank but ours to be 3x that
So that we have a distribution over our blank - we want the lowest signal we analyze to be above that but how much overlap in dist?
we ideally want just a 5% overlap and to do that we need 3.3 stdev - that means our distribution overlaps with the blank distribution such that the portion in the blank is our BETA rate - false negative
and the region ov overlap in our sample dist is alpha - false positive.
Basically want a 5% overalp between the 2 so often 2 *sd of blank or 3.3 uis used to achieve that - so 5% for type I and type II error (type I is in sample Type 2 is in blank
Old answer:
LOQ vs LOD
10x
Calculate LOD or LOQ from signal to noise
need to use it with a nother method to verify
its mean + either 3 or 10 * stdev
if linear cal curve its 3.3 or 10 * stdev / b
slope of linear regression
What are selectivity and specitificity
selectivity - abiltiy of method to determine analyte in complex matrix without interference
Specificity - confirm the method ability to assess the analytes in presence of any other components that might be present (including matrix)
so specificity is selectivity +
Accuracy vs precision
accruacy - trueness or bias - measure of systematic error compare to reference,
Precision -closeness of repeated individual measurements under specified conditions
How to run accuracy and rpecision tests
against standard material want accruacy within and between run - bias - use a low and high QC
Precision - use % CV
ROBUST what is it
capacity of method to be uanffected by natural variation - test over range of parameters
UNCERTAINTY
sig source must be identified and tabulated
2 types
A and B
A is random
B is systematic
example - user skill, sampling, environe , instrument, etc
Stability
use QC - store at room temp, 4 cetc test against fresh
HOW DO WE HANDLE NON DETECTS
Exclude or delete from data set (worst)
Substitue (0, 1/2 LOD , LOD etc
Left and right indicate whether its too low or too high in terms of an unknown
What is survival analysis and NADA
- how long will it be before event occurs (eg medical)
NADA is non detects and data analysis
Fit What is fit for purpose
ensures the analytical method fills certain criteria of reliability and can can perform - gives us confidence shows its reproducible and repeatable
(so as a list
reproducible, broad coverage - sensitivity and selectivity, linearity, precision, stable)
What are some key poitns to precision testing
sample should be stable and homogenous (representative of whats bein tested)
Should be applied to the whole sample preparation method analysis procedure
2 factors - precision estimate and design of precision experiment
What are the types of precision estimates
1) REPEATABILITY -within batch or intra assay - one analyst on the same equipment over a short time period
2) Intermediate precision - made in a single lab but variable conditions different days, analysts, equipment etc - within lab reproducibility
3) Reproducibility - DIfferent labs - different equipment (interlab)
Types of precision experiments
Simple replication - repeated measurements on a suitable sample - want 6-15 reps
NEsted design - used when cant generate enough reps with simple replication (not feasible) - basically each batch has different params - so can be inter lab, intra lab etc
PRECISION limits - what are and howto calc
repeatabiltiy limit = r = t* root(2) *s
confidence interal 95% for difference between two results obtained under repeatbility conditions
Reproducibility is R = troot(2)sr
t is 2 tailed students t tested for confidence level and DOF,
They are calculated by multiplying the repeatability standard deviation (sr) or the reproducibility standard deviation (sR) by 2.8 respectively. The factor 2.8 is derived from 1.96 (95% of the population is within 1.96 standard deviations of the mean) times the square root of 2.
How to statistically evaluate precision estimates
F test
What is bias and how calculated and how evaluated
difference form true value - so just mean - accepted - can be %
t test statistic
Ruggedness study - how evaluated/set up
PLACKET BURMAN -
7 parameters to study - you pick (eg extraction time) each has levels (eg 30 min extract vs 10 min)
to investigate effect - difference between average of results of parameter at normal level vs average of results at alternate level
Measurement of uncertainty - what is and how tested
dispersion of values possible for measurement - eg stdev
can be propgated
ROC curves
OK so a ROC curve is a plot of TP RATE against FP rate
we often take AUC - area under curve
AUC ranges in value from 0-1 - a model with 100% wrong predictions has an AUC of 0 and has an AUC of1 if 100% right
REceiever operating characteristics - evalute prediction accuracy of classifier model
- tradeoff between sensitivity and specificity - the same LOD vs blank curve
formula
TPR = TP / all tP
FPR = FP / all FP
area under cuvrve AUC is the thing
An ROC curve (receiver operating characteristic curve) is a graph showing the performance of a classification model at all classification thresholds. This curve plots two parameters: True Positive Rate. False Positive Rate
https://en.wikipedia.org/wiki/Receiver_operating_characteristic
What is metrology
formal system to enable informed decision through data assessment - levels of confidence to what we’re doing - reliable network of measurements for use to confidently make assessments about concentration
3 fundanmentals of metrology
1) TRACABILITY - SI - translates units to results - go from standards to where you are (higher order to CRM’s
2) UNCERTAINTY - (of measure) -using the rsd in results to make claims not individual points - the dist
3) VALIDATION (methods et)
What is QA and exmaples
he planned and systematic activities implemented in a quality system so that quality
requirements for a service will be fulfilled, quality assurance occurs before the data is collected
eg suitable lab environment, educated staff, training, documented and validated methods, preventative actions, etc
What is QC
Quality Control: the observation techniques and activities used to evaluate and report quality,
quality control occurs during and after data is collected
examples: blanks, spiked samples, controls, reference materials etc
Example of system suitabiltiy testing
small number of standards - acquire data for accuracy precision - not in bio, m/z accuracyRT peak shape are assessed
Dispersion ratio - what is it
Standard deviation for pooled QC sample vs test sample stdevso can use the D- ratio (MAD of QC/MAD of sample) and
AD of 0% means technical variance Is 0 - perfect measurement all cahgnes are due to biological cause
AD of 100% means all variance is due to noise - no bio info
Whats pooled QC
generated a single QC sample that can be distributed evenly throughout analytical batch
Batch design and pooled QC - what can you do
basically run QC throughout to see time based variance
Main uses of reference materials
examine skills of analyst
cOntrols
precision accuracy
accreditation
means uncertainty
How to score profiency testing results
2 steps - specify assigned value
and setting the standard dev
- ASSIGNMED VALUE - can either be known (CRM), REFERENCE value ( one lab determines) or it can be determined based off consensusfrom other labs
STDEV - set by scheme organizer - set by prescription or based on the results of a reproducibility experiment, from a general model (eg horowitz funct)
What is Z and Q in proficiecny testing
Z score is what we thinkg - value minus mean divided by stdev
z less than 2 (abs value of Z) pretty good - less than 3 - hmm questionable
Q score - alternative to Z - takes no account of stdev - dsit of Q centered on 0 then - relies on EXTERNAL PRESCRIPTION of acceptability
WHat is a YOUDEN plot
scatter plot - plots results from multiple labs on graph to show
if labs are equals, outliers, inconsistencies etc
x and y each represent one of the reported values (eg concentration of analytes A and B)
draw lines parallel to x and y axis and depending on where they are indicates - various things about results - eg random error vs systematic error
SHEWHART plot
sequential plots of observatiosn from QC material analyzed in succesiely - mean QC for each run and measurement # (y axis shows the mean
General princiiples of experimental design
Resaerch method where manipulate independant variables and look at dependant variable
things to do:
arrange experiments for cancellation or comparison..? - bias
plan to do replication or independent uncertainty estimates (precision)
Need statistical analysis or approach
Experimental designs list 4
Simple replication - series of observations on a single test material
Lienar calibration design - observations at a range of levels (some quantitative factor)
Nested: - has levels of factors in unique to that level
Factorial - has factors or levels but not wholly distinct - eg one group can be one factor, another can be another and and another group can be both factors at once
Why do we randomzie in expperimental design
to minimize nuisance effects - unwanted effects that influence the results - eg not effected by ordering/sampling order i
What is blocking
Basically have all replicates/groups of test materials subject to same nuisance effects (run at the same time - eg we have sets a b and c
What is blocking
Basically have all replicates/groups of test materials subject to same nuisance effects (run at the same time - eg we have sets a b and c - we can run them separately or run all in the same trial so subject o sam eeffects
Sampling theory (define randomization, representation and composite)
Randomization - equal membrs of pop - equal chance for selection
Representation - have enough of a population to draw inference on total pop
Composite - reduce effort by combining individuals to make a subset
List different sampling strats
Simple - everything equal chance (easy but not great for long continuous sequences also doesn’t reflect sub groups in population)
Stratified - divide pop into segments and randomly sample each segment - good because minimizes variance further - can get unique pockets
Systematic - First select random m then further ar at a fixed interval -simple and easy - regularly covers everything - cannot deal with any number specific variation - will miss it
4 quantities of power analysis
sample size
significance level (alpha 0 probabiltiy of making type I error)
Power - one minus the probability of making a type II error (probability of finding an effect is there
effect size - magnitude of the effect under alternate research hypothesis
How do you determine how many participants are needed for a study
power.t.test power package - theres a test you can do - uses sig level, power level etc can do for various tests, ANOVA
need means, common error variance etc, anova, linear regression chi squared etc
What is proportionality constant k
basically signal from instrumetn = the concentration * this factor
What is single point cal
basically just using this proportionality factor - just one point (S = k*C)- I guess also by default does through 0 then
Sensitivity from calibration curve
sensitivity is the slope b - capiabiltiy of responding reliably across changes in analyte concentreation
What is r in cal curve -
Its the pearson correlation coefficient - to describe relationship of response and concnvertation - 1- -1 describing correlation
R^2 measure how close data fits to linear model - 99% means 99% of difference variability in our responseis accounted for by changes in concentration
How to evaluate matrix effect
take sample matrix - extrat and spike sample in - compare to a normal standard solution (response/response) -1 ) - if neg value suppression
OR can do spiked recovery - compare matrix unspiked to matrix spiked (in same matrix) -
this is (spiked sample - unspiked ) / Cadded x100
Tyes of blanks -
method blank -unspiked sample
reagent blank -ust solvent
afield blank - unspiekd sample goes for trip (trip same but unopened)
Weighted regression
error with a emasruement proportional to conetration so with larger concentration more error so we give more weight to points where error bars are smallest for higher weights (divide by n -
Methods of standard addition
make cal curve in sample -
ISTD
strucutre nalogue, Stable isotop elabeled