Skip to content
Tweet Cruncher

Guided tour

The project in three short walkthroughs

Each video follows one workflow from start to finish, with the step shown on screen and as captions. A Playwright script recorded them from this site and checked every step on the way (the published numbers, the fitted serial fraction, the 8-rank run matching the 1-rank baseline), so the same seeds reproduce the same inputs. Timings are measured live and differ from machine to machine.

Walkthrough 1 of 3 · /results

The results

The dashboard of the three answers the MPI program wrote on Spartan in 2023: tweets per capital city on a map, the top tweeters, the city-hoppers heatmap and the raw result files.

Setup: No input needed: every number is transcribed from the 2023 submission.Open the MP4

Steps (transcript)

  1. 1Open Results: the three answers from the final Spartan run over 9,092,274 tweets
  2. 2Task 2: tweets per Greater Capital City, on a map of Australia and in a table
  3. 3Hover a city to highlight it on the map and in the table: Melbourne edges out Sydney
  4. 4Task 1: the ten most prolific authors, ties sharing a place
  5. 5Task 3: ten authors tweeted from all eight capitals; each cell is a city's share
  6. 6The three result files, laid out in main.py's format
Try it yourself

Walkthrough 2 of 3 · /scaling

Scaling lab

The Spartan benchmark jobs, their speedup and Karp–Flatt serial fraction, and an Amdahl's law explorer fitted to the measured runs, with the serial-fraction slider dragged to ask “what if”.

Setup: Final submission's jobs 46094405–07 (1 × 1, 1 × 8 and 2 × 4 cores, one run each); the explorer starts from the least-squares fit.Open the MP4

Steps (transcript)

  1. 1Open Scaling: the 1 × 1, 1 × 8 and 2 × 4-core Spartan jobs, one run each
  2. 2Speedup, efficiency and the Karp–Flatt serial fraction for every job
  3. 3Amdahl's law explorer: the serial fraction fitted to the 8-core runs, about 3.2%
  4. 4Drag the serial fraction f: the curve, the predictions and the ceiling follow
  5. 5Drag the workers n: predicted time and speedup at that core count
  6. 6Gustafson's law for contrast: the same f if the input grew with the cores
  7. 7Reset to the fit: one run per layout, so 3.2% is a point estimate with no interval
Try it yourself

Walkthrough 3 of 3 · /lab

MPI in your browser

A seeded synthetic file crunched by the ported algorithm with one Web Worker per MPI rank, a repeated benchmark across worker counts with interval estimates, then the optional bring-your-own-key question answering, shown with a mocked reply (no real key is used).

Setup: Seed 2023, 100,000 tweets. Benchmark: 1 warm-up and 5 timed rounds per worker count, round order seed 2023, bootstrap seed 90024; worker counts 1, 3, 4, 6, 8 and 10 on the 10-core recording machine (2 is skipped: the original cannot run on 2 ranks). Ask: a placeholder key and a mocked reply.Open the MP4

Mocked AI response for illustration. Steps 11 to 13 use a placeholder key. Requests to the provider are intercepted in the browser and answered by a mock, so no model was called: the answer shows how the feature labels, cites, checks and logs a reply, not what a real model would say.

Steps (transcript)

  1. 1Open the MPI lab: one Web Worker per MPI rank, running the ported algorithm
  2. 2Generate the seeded synthetic file: seed 2023, 100,000 tweets in bigTwitter.json's layout
  3. 3Run on 1 rank first: the clean baseline, like the 1 node × 1 core job
  4. 4Run on 8 ranks: each scans its own byte range, then ranks 0, 1 and 2 reduce the tasks
  5. 5Same three result files as the 1-rank baseline; the answers from the task ranks
  6. 6Benchmark: 5 timed rounds per worker count after a warm-up, in shuffled order
  7. 7Speedup with nominal 95% bootstrap intervals, the fitted Amdahl curve and its band
  8. 8Per worker count: median time with an exact interval, efficiency and Karp–Flatt
  9. 9Ask the results: optional, answered only from the result tables, with your own key
  10. 10AI settings: the key stays in this browser and goes only to the provider
  11. 11For this demo: a placeholder key, never a real one; provider calls are interceptedMocked AI response for illustration
  12. 12A mocked answer, labelled AI-generated, with cited rows and an automatic checkMocked AI response for illustration
  13. 13Accept it: the review is recorded in the AI audit log, with JSON and CSV exportMocked AI response for illustration
  14. 14Forget key: the placeholder is removed from this browser
Try it yourself

Screenshots

Every key feature at a glance

Captured by the same script, in light mode at 1440 × 900 (the landing page also in dark mode) and on a 390 px phone. Select one to enlarge it; the arrow keys step through the set.

Desktop · 1440 × 900

Mobile · 390 × 844

How these were made

pnpm showcase runs web/e2e/showcase.spec.ts on the system Chrome: it plays each journey at a human pace with an on-screen caption and cursor, asserts what it shows, and records it at 1280 × 800. ffmpeg then encodes the H.264 videos here and the GIFs in the README. The captions and the step lists on this page are the same text as the on-screen steps.

No real API key is used anywhere in these recordings. Where the AI feature appears, the key is a placeholder, every request to the provider is intercepted in the browser, and the reply is a labelled mock whose text starts with “Mocked response for illustration.”.