What this competency actually requires

Data Analysis is the professional capacity to inspect, clean, model, and interpret data without confusing calculation with evidence. Competence is visible when a practitioner can explain where the evidence came from, why the method fits the decision, which assumptions carry the conclusion, and how uncertainty should affect action.

The intellectual foundation includes Tukey’s exploratory tradition and the reproducible-analysis principles articulated by Peng (Tukey, 1977) (Peng, 2011).

Research foundations

Good analysis begins before a model is selected. A reproducible workflow therefore preserves source data, documents transformations, versions code or formulas, and separates exploratory work from confirmatory claims (Wickham, 2014).

Uncertainty has several layers. Sampling error is only one. They report practical magnitude and plausible ranges, examine exceptions, and identify conditions under which the recommendation would change (Gelman & Vehtari, 2020).

Ethical reasoning is part of analytical quality.

A defensible working method

Begin with a decision question and a data-generating-process map. Profile missingness, units, duplicates, ranges, and lineage before choosing a model. Visualize distributions and relationships, state assumptions, compare plausible specifications, and report uncertainty alongside effect size.

Before delivery, another qualified person should be able to reproduce the main result and challenge the strongest assumption. It should state limitations in language proportionate to the decision rather than burying them in technical notes.

1. Data Analysis framing: define the decision, outcome, unit of analysis, time horizon, and cost of error.
2. Audit: verify provenance, definitions, permissions, missingness, and measurement quality.
3. Analyze: establish a baseline, choose methods by assumptions, and retain a transparent trail.
4. Challenge: test alternatives, edge cases, sensitivity, and subgroup performance.
5. Act and learn: assign ownership, monitor consequences, and revisit the model when conditions change.

Failure modes and professional judgment

A polished dashboard can conceal selection bias, leakage, miscoding, or an outcome chosen after results were seen. In Data Analysis, these defects are consequential because they can redirect resources and distribute harm.

A mature practitioner actively searches for disconfirming evidence. When evidence is weak, the correct output may be a smaller decision, a reversible pilot, better measurement, or no recommendation yet (National Academies of Sciences, 2019).

Credibility grows from traceability rather than certainty theater.

Evidence of mastery

Evidence of Data Analysis mastery includes a well-scoped question, data dictionary, assumption register, reproducible analysis, appropriate validation, plain-language decision memo, and post-decision monitoring plan. It also shows restraint: the practitioner knows which conclusions are supported, which are provisional, and which remain outside the evidence (Cleveland, 1993).

For leaders, the practical test is whether the competency improves decisions over repeated cycles.

Questions for review

Before approving work in Data Analysis, reviewers should ask: Is the decision question explicit? Are the data and definitions fit for that question? Could another analyst reproduce the result? Were reasonable alternatives tested? Are uncertainty and limitations visible where the recommendation appears? Is the action proportionate to the strength of evidence? For Data Analysis, these questions turn quality assurance into a practical discipline rather than a ceremonial sign-off.

The review record should preserve the original question, source dates, exclusions, transformations, analytical choices, contrary findings, reviewer comments, and final disposition.

Evidence, controls, and decision use

Data analysis is a sequence of accountable choices about collection, cleaning, representation, comparison, and interpretation. Exploratory analysis should expose distributions, missingness, outliers, and unexpected relationships before a confirmatory claim is made (Tukey, 1977). A p-value does not report the probability that a hypothesis is true, the size of an effect, or the importance of a result; those require design knowledge, estimates, intervals, and context (Wasserstein & Lazar, 2016). Teams should preserve the source extract, transformation logic, exclusions, data dictionary, analysis code, and decision record so another analyst can reproduce the path from observation to conclusion. Segment results where averages could conceal operational or distributional differences, and distinguish descriptive summaries, causal estimates, and predictions because they answer different questions (Shmueli, 2010).

Applied review protocol

A practical analysis brief can be reviewed in six fields: decision question, population and period, measures and definitions, planned comparisons, uncertainty treatment, and action threshold. During review, ask an analyst to reproduce one reported number from the raw extract and explain every transformation. Then inspect a disaggregated view, an alternative reasonable specification, and the records excluded from analysis. Release the result only when the owner can state which conclusion is directly observed, which is estimated, and what evidence would reverse it. For recurring reports, automate validation checks for row counts, duplicates, ranges, missingness, schema changes, and reconciliation to a trusted total. This makes data quality visible before a decision depends on it.

Limitations and misuse

A clean chart cannot repair biased sampling, ambiguous definitions, measurement error, or data generated by a changing process. Searching many outcomes and reporting only favorable patterns inflates false discoveries. Analysts should label exploratory findings, document alternative specifications, report uncertainty and missing-data treatment, and test whether a conclusion changes under defensible analytic choices.