SEPIA  ·  ← roadmap  ·  all docs

SEPIA — Bug evidence log

SEPIA — Bug evidence log

Every issue in ../plan.md, with the exact commands to reproduce it, the actual output observed, and — added 2026-07-23 — the downstream consequence each error had on the analyses, figures, and conclusions. Verdicts are honest: one claim (A4) did not hold up and is downgraded; several bugs are shown to have had no downstream consequence, which is stated plainly.

On the consequence layer. The "Downstream consequences" blocks were produced by tracing each bug forward to a concrete artifact — a degenerate distribution in a committed output, a genome effectively demoted in a figure, a model on shaky footing, or a sentence in the thesis — and then adversarially re-deriving each claim from the actual source. Every consequence is tagged [DEMONSTRATED] (visible in a committed file / R page / thesis), [NONE] (honestly, no effect — the conclusion survives), or [LATENT] (a real effect that can only be confirmed on the cluster). Thesis page numbers are the printed numbers.

B=/nfs/turbo/lsa-tlasisi1/lheald_thesis/aDNA_data

⓵ / A2 — hg38 panel vs hg19 data (build mismatch) — CONFIRMED

Claim. The 222-SNP pigmentation panel is on GRCh38/hg38, but the reference genome and all sequence data (SGDP + archaic) are on GRCh37/hg19. So every pigmentation-SNP measurement is taken at the wrong coordinates.

Reproduce — the panel is hg38 (local; rs1426654/SLC24A5 is hg38 15:48134288, hg19 15:48426484):

grep -E "48134288|48426484" data/snps.txt          # committed 222-SNP panel
for p in 6:396322 16:89919710 15:28120473 5:33951589; do grep -q "^${p}:" data/snps.txt && echo "$p YES"; done
15:48134288::            ← hg38 SLC24A5 (NOT 48426484 = hg19)
6:396322 YES   16:89919710 YES   15:28120473 YES   5:33951589 YES   (all hg38)

Reproduce — the reference/data is hg19 (cluster):

ssh greatlakes "head -1 $B/reference/hs37d5.fa"
>1 dna:chromosome chromosome:GRCh37:1:1:249250621:1     ← GRCh37 / hg19

Reproduce — SGDP is monomorphic at the hg38 positions (cluster; afreq built by pig_pca.slurm):

ssh greatlakes "awk 'NR>1{n++; if(\$5+0==0)z++} END{print \"sites=\"n\"  ALT_FREQ==0: \"z}' $B/pca/pigmentation/pigmentation_snps_freq.afreq"
sites=222  ALT_FREQ==0: 221     ← only 1 of 222 sites carries any variation

Verdict: CONFIRMED. The single biggest issue; the reason B2 (normalize to hg38) is a correctness fix.

Downstream consequences (verified 2026-07-23):


Projection collapse — ancient pigmentation PCA stacks all samples on one point — CONFIRMED

Claim. The PLINK2 --score projection of the archaic genotypes onto the modern pigmentation PCA produces a single point repeated for every sample.

Reproduce (local):

awk -F',' 'NR>1{print $5}' data/ancient.projected.pig.sscore | sort -u        # distinct PC1_AVG values
0.0335547        ← exactly ONE value for all 15 archaic samples

Verdict: CONFIRMED. Two compounding causes: (1) the near-monomorphic panel from the build bug (rank-1 substrate), and (2) the --score step mean-imputes missing genotypes to the reference-frequency mean without no-mean-imputation, so with high archaic missingness every sample maps to the reference centroid. Fix in Q5: add no-mean-imputation, and harmonize the build (B2).

Downstream consequences (verified 2026-07-23):


A1 — Denisova 3 chromosome-naming mismatch — CONFIRMED

Claim. Denisova 3's BAM uses chr-prefixed contigs while every other sample, the reference, and the panels use bare names, so its extraction fails and it looks empty.

Reproduce (cluster):

ssh greatlakes "module load Bioinformatics samtools/1.21;
  samtools view -H $B/results/filtered/filtered_Denisova3.bam | grep '^@SQ' | head -2;
  samtools view -H $B/results/filtered/filtered_vindija33.bam | grep '^@SQ' | head -2"
D3:    @SQ  SN:chrM ...   @SQ  SN:chr1 ...     ← chr-prefixed (only sample that is)
VI33:  @SQ  SN:1 ...      @SQ  SN:2 ...        ← bare (matches reference & panels)

Verdict: CONFIRMED. Independent of the build issue (this is why D3 specifically is empty). Fix = harmonize contig names (reuse the reheader in pca_ancient.slurm), then re-run.

Downstream consequences (verified 2026-07-23):


A3 — Whole-genome ancient calling is haploid (--ploidy 1) on autosomes — CONFIRMED

Reproduce (cluster):

ssh greatlakes "grep -nE 'ploidy|CHRS=' $B/ancient_wg.slurm;
  module load Bioinformatics bcftools;
  bcftools view -H -r 1 $B/ancient_wholegenome/ancient_sgdp_wg.vcf.gz | head -3 | cut -f1,2,4,5,10-12"
64: CHRS=( {1..22} X Y )      ← autosomes included
83:     --ploidy 1 \
1  846808  C  T  0:0,255:27,0   .:0,0:0,0   0:0,67:2,0     ← GT is a single allele, never 0/1

Verdict: CONFIRMED. Autosomes are called haploid, so heterozygotes are never emitted.

Downstream consequences (verified 2026-07-23):


A4 — "bcftools call -mv drops non-variant sites" — DOWNGRADED (not an active bug)

Reproduce (cluster):

ssh greatlakes "grep -nA1 'bcftools call' $B/pigmentation_ancient_call.slurm $B/pig_call_miss.slurm;
  module load Bioinformatics bcftools;
  bcftools view -H $B/pca/pigmentation/ancient/ancient_pigmentation.vcf.gz | wc -l"
pigmentation_ancient_call.slurm:  bcftools call -mv     ← variants-only
pig_call_miss.slurm:              bcftools call -c      ← keeps all sites
records in ancient_pigmentation.vcf.gz: 222             ← FULL panel, nothing dropped

Verdict: NOT AN ACTIVE BUG. The committed output has all 222 sites, so the keep-all (-c) script is the one used. Cleanup only (delete the unused -mv script).

Downstream consequences (verified 2026-07-23):


A5 — Orphan / broken scripts — CONFIRMED

Reproduce (cluster): ancient_merge.slurm's glob *.wg.vcf.gz matches nothing (real file is ancient_sgdp_wg.vcf.gz); pca_ancient.slurm references a missing reference/hg19.fa and its pca_snps output is used nowhere.

Verdict: CONFIRMED. Both are cleanup/deletion candidates.

Downstream consequences (verified 2026-07-23):


Consequences confined to the published workflowr site (they did not propagate into the thesis, which is correct):


Depth missingness model — redundant random effect, stale prose, unseeded MC — CONFIRMED (with corrections)

analysis/depth.Rmd fits glmer(Missing ~ factor(Chromosome) + age_scaled + (1|Coverage) + (1|Sample)).

Downstream consequences (verified 2026-07-23):


subset_sgdp region-string bug + 1-based/0-based coordinate split — CONFIRMED (latent — no reached-result impact shown)

From pipeline_workbook.md: subset_sgdp.slurm writes CHR:from-to region strings then feeds them to bcftools view -R FILE (which parses only tab-delimited CHROM/POS or BED); separately sgdp.snps.pos is used 1-based in one script but converted to 0-based BED in another.

Downstream consequences (verified 2026-07-23):


Summary

ID Claim Verdict Worst demonstrated downstream consequence
⓵ / A2 build hg38 panel vs hg19 data → wrong coordinates CONFIRMED Rank-1 pigmentation PCA; 221/222 SNPs dead; Fig 8 + abstract conclusions are one-SNP artifacts
projection collapse --score mean-imputation + rank-1 panel CONFIRMED All 15 archaics identical in Fig 8; "cluster with Africans" rests on one collapsed point
A1 Denisova 3 chr-naming mismatch CONFIRMED A 30× genome recovers at 0.5×/69-of-222; misplaced in Figs 5 & 7 vs its own Table 3 class
A3 --ploidy 1 haploid autosomal calling CONFIRMED Haploid archaics scored on a diploid basis; WG-PCA conclusion nonetheless survives
A4 -mv drops non-variant sites DOWNGRADED None — committed output has all 222 sites (ALLELE_CT=444)
A5 Orphan/broken scripts CONFIRMED None — no committed artifact or figure traces to them
A6 Website prose/link/license/reproducibility CONFIRMED "395 SNPs" mislabel; empty LICENSE; site cannot rebuild (missing CSVs)
depth model Redundant RE, stale prose, unseeded MC CONFIRMED Coverage variance from 2 groups; prose/thesis over-state chr5; MC is actually seeded
subset/coord -R region-string + 1/0-based split CONFIRMED (latent) No shown impact on reported results; branch identity unresolved without the cluster