How I made Rustdoc 33% faster in one week

22 points by rrampage

junon

This is awesome. Curious on the profiling setup there; flame graphs are something I've been after and haven't found a clear way to get them. I'd love to know how it was done.

lalitm
Not the author and might be totally wrong but think they might have used:
1. -Z self-profile [1] compile flag to collect the data
2. crox in the measureme [2] toolkit to convert to a Chrome JSON trace
3. visualized the resulting trace with Perfetto [3] which has support for showing the timeline and you can select any region of time of the trace to aggregate it into a flamegraph.
Not sure if the author is here but I'd love if they confirm whether this is the case.

[1] https://rustc-dev-guide.rust-lang.org/profiling.html

[2] https://github.com/rust-lang/measureme/blob/master/crox/README.md

3] https://ui.perfetto.dev and https://docs.perfetto.dev
- junon
  
  Thanks, at the very least this is a good jumping off point. I appreciate it!
kaycebasques
before each stable version of Rust is published, the release team runs a tool called Crater that tests the new version across the public Rust ecosystem

Great idea. Haven't heard of any other docs engines actively validating new releases against their ecosystem. The status quo is more along the lines of "we have a release candidate, please try it and report bugs." How much of the ecosystem does Crater test against?

Also pleased to hear the mention about runtime GUI testing via Puppeteer.

This post got me wondering whether anyone has created rustdoc internals content. Sure enough:
- https://rustc-dev-guide.rust-lang.org/rustdoc.html
- https://rustc-dev-guide.rust-lang.org/rustdoc-internals.html
Tangential rustdoc question. Has the JSON output stabilized? Last I checked, it was experimental and had a bunch of compiler internals info. Use case for me (a technical writer) is basically connecting the Sphinx, Doxygen, and rustdoc parts of our website together more cohesively. Doxygen provides tagfile and XML output. Sphinx has extension API that lets you inspect and extract whatever metadata you need. Didn't find a good rustdoc solution along these lines.
- Mateon1
  
  As far as I can tell (I don't follow Rust development particularly closely, but I have seen the processes unfold across multiple large, and some small PRs), crater runs across ~all crates on crates.io. This is not only used for stable releases, it's often used for running experiments for PRs that may visibly change compiler behavior and cause potential breakage. I've seen this the most in various lifetime related PRs, more recently the trait solver changes, and I'm pretty sure I've seen crater runs for some MIR changes, but I don't recall the details.
  
  EDIT: actually just cross-checked the numbers for crates.io and a recent crater run, it seems to build significantly more crates than just crates.io (324k crates vs 1.08M builds), and the reports do include a lot of Github crates.
  With some slight digging, it seems crater uses https://github.com/rust-lang/rust-repos as a source for public Github Rust repos
  - ChrisDenton
    
    Yes, it builds everything on crates.io and rust repos on Github. There are currently some limitations and exclusions though. The crater runs have finite resources and running a lot of cargo instances in parallel can be resource intensive so sometimes building crates will spuriously fail. Rerunning helps there. There are also crates that require special environments or specific cfgs to run tests. Making a general solution would be more complicated, maybe impossible without manual tweaking. Also crater only checks x86_64-linux. Crater could technically run on other platforms but it's already fairly expensive doing it for one platform. Maybe adding cross-checking would be a way to (relatively) cheaply catch more issues.
    
    Those caveats aside it is impressive and does catch most issues.
- ssokolow
  
  How much of the ecosystem does Crater test against?
  
  It depends. At maximum, all publicly accessible crates, but they can choose on a run-by-run basis. (eg. to efficiently iterate on a proposed language change's effect on known-problem crates identified in a previous run so they can minimize disruption from something allowed by the stability promise, like changes to type inference.)