Draft: work in progress. This recipe is not finished yet; sections may be incomplete and results may change.

Status: draft (2026-11-17), sections 1-10. Run with migrate-n 7.0.114; the two-population runs with M = 5 used 7.0.114 plus the fix for population-mlalpha={1 1}, released in 7.0.115 and 6.1.91. Migrate 6.1.91 and newer (5.0.7 tree) run the same model; use 6.1.91/7.0.115 or newer for a shared alpha.

The Kingman coalescent assumes exponential waiting times between events. The Mittag-Leffler (fractional) coalescent replaces them with heavy-tailed waiting times governed by a parameter alpha (alpha = 1 is Kingman). This recipe simulates data under the fractional coalescent, estimates alpha in one and two populations, shows when alpha can and cannot be estimated, and compares the fractional and the Kingman model with marginal likelihoods. Read 00-basics, 01-migration and 05-model-selection first.

Contents

  1. The Mittag-Leffler coalescent
  2. The data and the true values
  3. Mittag-Leffler in the parmfile
  4. One population
  5. Two populations with strong migration (M = 50)
  6. Two populations, M = 5, 10 loci
  7. More data: 30 loci
  8. Model choice
  9. Practical advice
  10. What next

1. The Mittag-Leffler coalescent

In the Kingman coalescent the waiting time to the next event (looking backward in time, lineage view) is exponential. Mashayekhi & Beerli (2019) replaced it with the Mittag-Leffler distribution: its survival function is E_alpha(-lambda t^alpha), where E_alpha is the Mittag-Leffler function and lambda the rate of the events. With alpha = 1, E_1(-x) = exp(-x) and the Kingman coalescent is back. With alpha < 1 most waiting times are short and a few are very long (the tail decays like a power of t, not exponentially). Genealogies of such populations have many quick coalescences and some very long branches. The model was proposed for populations in which reproductive success varies strongly among individuals and over time (heterogeneous environments).

In Migrate each population runs its own clock with its own alpha. The clock of population i carries all events of its lineages (backward in time): coalescence at rate k(k-1)/Theta_i with k lineages, and migration of a lineage out of i (into i from the other population, forward in time). When the number of lineages of a population changes, its clock restarts.

Use it when the Kingman model does not fit (section 9 lists the hints), and always test it against Kingman (section 8): alpha needs many loci.

2. The data and the true values

scripts/simulate_ml.py simulates genealogies under the model of section 1, backward in time: each population draws a Mittag-Leffler waiting time with the total rate of its lineages (coalescence plus migration out, lineage view), the population whose clock rings first has the next event (coalescence or migration, in proportion to their rates), and every population whose number of lineages changed restarts its clock. Sequences evolve along the genealogy under JC69. With alpha = 1 this is the structured Kingman coalescent. The truth of each data set is in data/*/truth.json:

DataPopulations x sequencesLoci x sitesThetaMalphaSeed
data/onepop-a071 x 2010 x 5000.01-0.7801
data/onepop-a101 x 2010 x 5000.01-1.0802
data/twopop2 x 1010 x 5000.01, 0.0150, 501.0, 0.7803
data/twopop-m52 x 1010 x 5000.01, 0.015, 51.0, 0.7805
data/twopop-m5-l302 x 1030 x 10000.01, 0.015, 51.0, 0.7806

Population 1 follows Kingman (alpha_1 = 1), population 2 is fractional (alpha_2 = 0.7). All data use the time scale LOCUS (section 3). The commands that made them are in manifest.json, for example

python3 scripts/simulate_ml.py data/twopop-m5 --n 10 10 --theta 0.01 0.01 \
    --M 0 5 5 0 --alpha 1.0 0.7 --loci 10 --sites 500 --seed 805 --scale LOCUS

(--M, row by row: the backward rate of a lineage moving from i to j, which is M_j->i, migration into i from j forward in time.)

3. Mittag-Leffler in the parmfile

Three lines control the model:

LineMeaning
mittag-leffler-alpha=NOKingman coalescent (the default)
mittag-leffler-alpha=YESestimate alpha (same as YES:ESTIMATE)
mittag-leffler-alpha=YES:0.7fixed alpha (values 0.01 to 1.0; not used in this recipe)
population-mlalpha={0}no population has an alpha of its own: Kingman
population-mlalpha={1}all populations share one alpha, ML-alpha_1
population-mlalpha={1 1}the same, written out for two populations
population-mlalpha={1 2}each population has its own alpha
population-mlalpha={1 0}population 1 fractional, population 2 Kingman (not used here)
mittag-leffler-scale=LOCUSthe time scale of the clocks (below)

population-mlalpha works like population-growth (03-growth): one label per population, equal labels share a parameter, 0 means none. A shared alpha needs 6.1.91/7.0.115 or newer (older versions crashed in the Joint combination).

The time scale decides how the rate lambda enters the clock (survival function of the waiting time t, branch lengths in substitutions per site):

mittag-leffler-scaleSurvivalNote
SITEE_alpha(-lambda t^alpha)the clock of versions before 6.1.79/7.0.103
LOCUSE_alpha(-lambda c^(alpha-1) t^alpha), c = sites of the locustime counted in substitutions per locus
INVARIANTE_alpha(-(lambda t)^alpha)the default

For alpha = 1 all three are the Kingman coalescent; for alpha < 1 they are different models and give different Theta and alpha. Simulate and analyse with the same scale, and compare runs only with the same scale. This recipe uses LOCUS for both.

The prior of alpha is uniform from 0.3 to 1.0 (the outfile lists it as Population ML-alpha (ML-alpha_1)); the parmfiles keep it. Alpha is updated with the other parameters; the scaler move (Theta x c^alpha, M / c^alpha) is on by default with Mittag-Leffler.

scripts/make_parmfiles.sh [migrate-n binary] writes all parmfiles from Migrate’s default parmfile, changing only the data model (JC69), the three lines above, the seed, the output files and heating=YES:1:POWER:5:

Parmfilemittag-leffler-alphapopulation-mlalpha
parmfiles/parmfile.kingmanNO{0}
parmfiles/parmfile.mlYES{1}
parmfiles/parmfile.twopop-kingmanNO{0}
parmfiles/parmfile.twopop-sharedYES{1 1}
parmfiles/parmfile.twopop-ownYES{1 2}

Run them in a directory with the infile (mpirun -np 11 migrate-n-mpi parmfile -nomenu). The one-population and the M = 50 runs use the parmfiles as they are (10000 samples, burn-in 10000, 5 chains). All M = 5 runs use long-sample=40000 and burn-in=20000, the 16-chain runs also heating=YES:1:POWER:16.

The screen shows many warnings like construct_tymelist root moved: new time = 10339.607162 (29 to 322 per run here). They are expected with alpha < 1, not an error: a very long waiting time can move the root of the genealogy far back.

4. One population

onepop-a07 (alpha 0.7) and onepop-a10 (alpha 1.0), each with the Kingman and the Mittag-Leffler model (11 MPI processes, 3 and 5 min). Medians and 95% HPD intervals:

DataModelRowTheta_1ML-alpha_1
a07KingmanJoint0.0302 [0.0249-0.0355]-
a07KingmanAll0.0114 [0.0060-0.0149]-
a07Mittag-LefflerJoint0.0131 [0.0091-0.0177]0.70 [0.56-0.84]
a07Mittag-LefflerAll0.0114 [0.0055-0.0181]0.94 [0.76-1.00]
a10KingmanJoint0.0115 [0.0090-0.0140]-
a10Mittag-LefflerJoint0.0115 [0.0089-0.0143]0.97 [0.90-1.00]
a10Mittag-LefflerAll0.0119 [0.0087-0.0155]0.97 [0.89-1.00]

True: Theta 0.01; alpha 0.7 and 1.0.

  • The Joint row recovers alpha = 0.7, and for the Kingman data it puts alpha against 1. The All row gives 0.94 for the fractional data: it multiplies the per-locus marginal posteriors of each parameter on its own, but within a locus alpha and Theta trade off against each other, and only the Joint combination keeps them together. Read alpha from the Joint row.
  • The Kingman model fitted to the fractional data gives a Joint Theta three times the truth: the few very long waiting times can only be explained by a large population. Its All row (0.0114) disagrees with its own Joint row; for the Kingman data the two rows agree (0.0114 and 0.0115). Two rows that disagree that much are a sign that the model does not fit.
  • On the Kingman data the fractional model costs nothing in the estimates: Theta is the same as with Kingman.

5. Two populations with strong migration (M = 50)

twopop: Theta 0.01 in both populations, M = 50 in both directions, alpha_1 = 1.0, alpha_2 = 0.7. Joint rows (* = Joint* row, large genealogy-sampling error, 01-migration); own-long is the own model with 40000 samples:

ModelTheta_1Theta_2M_2->1M_1->2ML-alpha_1ML-alpha_2BTI
truth0.010.0150501.00.7
Kingman {0}0.0120*0.006439*115*---10170.5
shared {1 1}0.0112*0.0048*43*1720.83*0.83*-10170.8
own {1 2}0.01140.0044382180.780.81*-10175.5
own-long0.0102*0.005047*156*0.77*0.90*-10204.6

Run times 16 to 29 min (own-long 63 min) on 11 processes.

  • The own model does not find which population is fractional: own-long puts the smaller alpha on population 1 (0.77 [0.64-0.90]) and the larger on population 2 (0.90 [0.75-1.00]), the reverse of the truth.
  • All models underestimate Theta_2 (0.0044-0.0064 instead of 0.01) and overestimate M_1->2 (2 to 4 times).
  • The marginal likelihoods (joint row) favour Kingman or tie with it (TI, BTI and SS agree on that).

Is the two-population setup itself wrong? The same Kingman parmfile on data simulated with alpha = 1 in both populations (seed 804, otherwise the same) gives Theta 0.0116 and 0.0120, M_2->1 72 [30-131], M_1->2 50 [12-97]: Theta and M are recovered, and the migration direction convention is right.

The reason is the strong migration. With M = 50 (Theta x M = 0.5, ten times the value of section 6) the lineages move between the populations often, the deep part of every genealogy is spent in both, and the two clocks act on the same lineages. The data then show a mixture of the two waiting-time distributions, and the per-population alphas cannot be separated. With strong gene flow, estimate one shared alpha at most.

6. Two populations, M = 5, 10 loci

twopop-m5: the same as twopop but with M = 5. 40000 samples, with 5 chains (local, 5 or 11 processes) and with 16 chains (HPC, 11 processes). Joint rows:

ModelChainsTheta_1Theta_2M_2->1M_1->2ML-alpha_1ML-alpha_2
truth0.010.01551.00.7
Kingman50.01190.01333.06.3--
shared50.01080.00995.011.00.870.87
own50.01100.00955.013.00.94 [0.83-1.00]0.79 [0.63-0.93]
Kingman160.0119 [0.0087-0.0156]0.0130 [0.0097-0.0171]3.0 [0-7.3]6.3 [1.3-10.7]--
shared160.01080.00975.011.00.87 [0.77-0.96]0.87
own160.0111 [0.0075-0.0147]0.0094 [0.0057-0.0133]5.7 [0-12.7]11.7 [2.7-26.7]0.95 [0.83-1.00]0.77 [0.61-0.92]

Run times with 16 chains: Kingman 29 min, own 165 min, shared 180 min.

  • With M = 5 the own model puts the alphas in the right order, and both intervals contain the truth; but the intervals overlap, so ten loci of 500 sites do not show that the populations differ.
  • The shared model gives a compromise (0.87).
  • The fractional models overestimate M_1->2 (11-13), with intervals that contain 5. The Kingman model gets Theta_2 a third too high.
  • The estimates hardly change between 5 and 16 chains; the marginal likelihoods do (section 8).

7. More data: 30 loci

twopop-m5-l30: the same model with 30 loci of 1000 sites (3 times the loci, twice the length), 16 chains, 40000 samples, 31 processes on the HPC. Joint rows:

ModelTheta_1Theta_2M_2->1M_1->2ML-alpha_1ML-alpha_2
truth0.010.01551.00.7
Kingman0.0144* [0.0119-0.0172]0.0502* [0.0418-0.0595]4.3 [2.7-6.7]2.3* [0.7-4.0]--
shared0.0102 [0.0083-0.0121]0.0140 [0.0111-0.0167]9.7 [4.0-15.3]4.3 [1.3-9.3]0.84 [0.79-0.89]0.84
own0.0109 [0.0091-0.0127]0.0129 [0.0102-0.0157]6.3 [2.7-10.0]4.3 [0-10.0]0.98 [0.93-1.00]0.72 [0.64-0.79]

Run times: Kingman 69 min, shared 288 min, own 329 min.

  • The own model now separates the populations: alpha_1 0.98 [0.93-1.00], alpha_2 0.72 [0.64-0.79]; the intervals no longer overlap and both contain the truth. The All row is also close here (0.97 and 0.67), but read the Joint row.
  • The shared alpha (0.84) is a compromise that fits neither population; its M_2->1 (9.7) is higher than with own alphas (6.3).
  • Theta_2 is a little high in both fractional models; its intervals just miss 0.01 (lower bound 0.0102 and 0.0111).
  • The Kingman model explains the long waiting times of population 2 with a Theta_2 five times the truth (Joint* row), and the outfile warns “Upper prior boundary seems too low” for Theta_2 at four loci (prior 0-0.1). Three of its four Joint rows are starred: the Kingman model fits these data so badly that its joint combination is unreliable.

8. Model choice

Log marginal likelihoods, “All … (joint)” line, minus the Kingman model (positive favours the fractional model):

DataChainsModelTIBTISS
twopop-m5 (10 loci)5shared-30.0+3.6-105.7
5own-46.6-1.7-171.7
16shared-4.4-4.1-3.8
16own-4.8-4.6-3.1
twopop-m5-l30 (30 loci)16shared+236.0+238.3+225.1
16own+254.4+256.3+210.6

The values of every run (joint and independent lines) are in manifest.json.

Five chains are not enough. With 5 chains TI, BTI and SS disagree by up to 170 log units and even in sign. With 16 chains they agree. The heated chains of a fractional model have to bridge from the posterior to the prior, where alpha and Theta take extreme values and waiting times become very long; a short ladder misses that part. The one-population runs (5 chains) show the same: on a07 (alpha 0.7) the fractional model is ahead by 50.5 (BTI) but behind by 8.3 (TI); on a10 (alpha 1) BTI and TI favour Kingman (13.0 and 51.8) but SS the fractional model (4.9).

Ten loci of 500 sites: with 16 chains all three estimators favour Kingman by about 4 log units, although population 2 is fractional; shared and own tie (differences below 1). The data do not contain enough information to pay for the extra parameters.

Thirty loci of 1000 sites: the answer reverses. Both fractional models are ahead of Kingman by more than 200 log units with all three estimators. The Kingman joint marginal likelihood has a large Monte Carlo error (10.0, against 0.6-0.7 for the fractional models; section 7), but the decision does not depend on it: the “(independent parameters)” line, which does not use the joint combination, gives the fractional models +65 to +111.

Shared or own alpha? TI and BTI favour own by 18 (joint line; 12-13 on the independent line), SS favours shared by 15 (20). With 16 chains the estimators agree on the large difference to Kingman, but not on this small one; the marginal likelihood does not decide it here. The estimates do: the own model gives two alphas with intervals that do not overlap, and its alpha_1 sits at 1.

These are single data sets; the 30-locus data differ from the 10-locus data in the number and the length of the loci and in the seed.

9. Practical advice

  • Read alpha in the Joint row. The All row pulls alpha toward 1.
  • Start with Kingman, then one shared alpha ({1}), then one alpha per population ({1 2}), and compare with marginal likelihoods from the “All … (joint)” line.
  • Use 16 heated chains (heating=YES:1:POWER:16) for model choice with Mittag-Leffler, and check that TI, BTI and SS agree. If they do not, the ladder is too short.
  • Alpha needs much data. Ten loci of 500 sites estimate alpha but do not make the Bayes factor favour the fractional model; thirty loci of 1000 sites do.
  • With strong migration (here M = 50, 4Nm = 0.5 into each population) per-population alphas are not identifiable; estimate one shared alpha.
  • A Kingman analysis of fractional data gives too large Theta values (3 to 5 times here), Joint and All rows that disagree, and prior-boundary warnings for Theta. These are hints to try the fractional coalescent.
  • Keep the same mittag-leffler-scale in all runs you compare, and state it when you report alpha and Theta.
  • Fractional runs take 2 to 6 times as long as Kingman runs.
  • The “construct_tymelist root moved” warnings are expected.

10. What next

  • 05-model-selection: marginal likelihoods and heating ladders in detail.
  • 03-growth: the other way to change the coalescent rate through time; population-mlalpha and population-growth use the same syntax.
  • A coverage study (many data sets per size) would show how often the Joint interval of alpha covers the truth and how many loci the Bayes factor needs.

References

  • Mashayekhi S, Beerli P (2019) Fractional coalescent. PNAS 116: 6244-6249. doi:10.1073/pnas.1810239116

Available Materials