One File - What if your lockfile and your package list were the same file?

13 points by aae

lilyball

I can see the appeal of this, but the realized profile doesn't include what was originally requested, and so there's no way to take a realized profile and say "now update to something newer". The realized profile doesn't tell you if the user originally wrote package "make" or package "make" version="4" (which I'm assuming here is valid input) or what. So now if the user wants to be able to update to something newer they have to keep around both files after all.

jhuntwork

Hmm, also, just to make one point possibly clearer, the file doesn't get updated in place after realization. At the moment, the actual realized profile.kdl is dropped inside the profile root, so from that standpoint you do get a new, separate file. The source input may have been one in your home or project dir and stays just as you defined it originally. The "bidirectional" aspect, or "one file" idea is that you could take that realized profile from the profile root and apply it into a new, different profile, perhaps on a different machine as real input.

There's definitely room for improvement though, and I expect real usage over time will show clearly where the pain or challenges are.
jhuntwork

Good feedback, thanks. I think that might be possible to incorporate. At least worth taking at look at.
- wink
  
  I was also confused why the calculated output is also under profile and not under a second top-level category (which might not be the best solution, but maybe the simplest)
bkhl

This is how Go does it and it's fine I'd say.
- technomancy
  
  Maven too. Never quite figured out why anyone would do it differently. Ironic that this person landed on mostly the right answer but had to rediscover it from first principles. (I say mostly because supporting ranges is a mistake, but a common one)
  - mxey
    
    I think the reason to do it differently is to distinguish the dependencies you need explicitly vs the transitive ones. If the need for a transitive dependency goes away, I don’t want to keep pulling that in.
    
    The answer to that is automatically managing the list of dependencies based on your imports, which is what “go mod tidy” does. I never manually add a library to go.mod. I add an import statement in my code and run the tidy command.
    
    Does Maven have something like this?
  - jhuntwork
    
    article author here - mere is new, so it's still finding its full shape. Suggestions for improvement are very welcome. I'll be providing a little usage doc in the very near future that will make it easier to test and provide feedback. It's a static binary with a dedicated store, so it should be easy to test on a vm or existing system.
    
    jmillikin
    
    This is a minor suggestion, but for your checksum format I recommend including the hash name.
    
    So instead of:
    
    package "make" version="4.4.1" release=3 \ content-hash="9426fcd11c446be26544b763d6d649186ec9fd98142204a79d582dc6409ecead"
    
    Something like:
    
    package "make" version="4.4.1" release=3 \ content-hash="sha256:9426fcd11c446be26544b763d6d649186ec9fd98142204a79d582dc6409ecead"
    
    There's also Subresource Integrity (SRI) which is used by the web ecosystem, it's more compact (base64 instead of hex) but a little unfamiliar-looking:
    
    package "make" version="4.4.1" release=3 \ content-hash="sha256-lCb80RxEa+JlRLdj1tZJGG7J/ZgUIgSnnVgtxkCezq0="
  - deevus
    
    Zig too
- viraptor
  
  It feels like it would be hard to preserve comments in this solution. With package list I can make an extensive comment about why something exists in a way it does. But something that rewrites the file will likely drop that.
  - muvlon
    
    This is a tricky programming problem, but not unsolvable. In the Rust ecosystem, most things use toml_edit when modifying Cargo.toml, which does preserve comments.
- diktomat
  
  I think that sounds great in theory, but fails in practice. Why would I want dozens or hundreds of dependencies in the file I edit by hand after resolving the first time? How would I update just one package: up the version of it, remove the hash, then track down all dependencies that need updating because of it and do the same? Solution to both is that it doesn’t update in place as you write, but what then is the advantage of two files of the same syntax in different places over two different files in the same directory? In practice, these files rarely travel independent of the project directory they’re in…
  
  I like your choice of kdl though!
- chriswarbo
  
  My initial thought when I read the title was "yes, like Nix"; but then I see Nix listed as having separate "lock files"? So it turns out that it's to do with flakes (yet another reason to avoid them, I suppose!)
  
  I've been transitioning my personal projects to specify external references via Git tree SHAs, since that allows fetching (ideally directly from IPFS; or otherwise from a designated source, like an existing clone, or online "forge", etc.) and verifying, without requiring two separate hashes.
  
  Ideally, if we want some external references, then we should try to only have one, that we update deliberately; and that will give us an "index" of everything else we might want. A good example is in the Haskell ecosystem, where we can specify a particular revision of the all-cabal-hashes repo (unreadable, but allows fetching+verifying); or a timestamp of the Hackage index (readable, but needs a separate hash); or a Stackage release (ditto). That way, a project can e.g. specify 2025-01-01T00:00:00Z as its "lock", and a deterministic version solver can choose from the packages that contains (all with their associated SHAs); if a dev bumps that "lock", e.g. to 2026-01-01T00:00:00Z, then it's a deliberate choice which can be reviewed and critiqued, unlike the diffs of auto-updated lock files with a bajillion hashes.
  
  (For those using Haskell with Nix: Nixpkgs already contains a pinned/locked Hackage version, so as long as we pin/lock a Nixpkgs revision, we get everything in Hackage too. That's how I do Haskell dependency solving in Nix)
  
  I wrote up some of my thoughts in this area a while ago.
- jaredkrinke
  
  I’d never heard of Mere before. It’s both musl libc-based and uses something like the Nix store? That sounds great!
  
  I had been dreaming of building something similar, just based on a smaller compiler than Clang (my main motivation being reproducible builds/environments without as huge of a required base footprint). Like a marriage of Oasis Linux and Nix (flakes). Note that I’m not sure my idea has actual practical value. I just like minimalist software.
- kryptiskt
  
  I'm pretty sure that this will end up with preople wanting to preserve their original config before resolution so they can recreate it from scratch, and will save a copy with a suffix like ".unlocked" or ".original". So the end result will be two files anyway.
- max-headroom
  
  This is probably annoying to work with when you're using an editor that doesn't automatically reload the file on external changes. Other than that, I feel like Nix already gives you the same guarantees despite using 2 files to achieve it. Interact with a local unlocked flake, Nix will lock it. Interact with a remote unlocked flake, Nix will use the latest versions of all the inputs. Interact with a flake that has oudated lock information, Nix will treat the relevant inputs as if they are unlocked.