Skip to main content
Huy's site

[Update] IPWHL: Maybe the real cheeses are the packages we helped along the way

So, according to a recent announcement in the mailing list, I now co-maintain the IPWHL project, also known as Floating Cheeses (I prefer the latter for it being more playful and pronounceable, but IPWHL is just quicker to type). So, I feel obliged to provide a more thorough introduction.

Les Cheeses

In short, IPWHL is a PyPI alternative (though, initially the package database would be collected from there). What it provides:

  • Decentralization
  • Security
  • Reproducibility

Decentralization

IPWHL uses IPFS for storing packages. This provides several advantages:

  • No single point of failure
  • Easy to mirror
  • Faster download thanks to P2P

As there have been several incidents of PyPI outages, this is a strong reason to use our cheeses.

Security

No single point of failure is a security feature itself, but besides that, IPWHL also is more secure because:

  • We have CIDs and we cryptographically sign the packages thanks to merkle dag
  • We avoid packaging packages that are typosquat attacks

Reproducibility

IPWHL has a pre-resolved dependency and its packaging strategy is similar to NixOS, a distro known for its reproducibility. The installer can uses the CID and the package name to reproduce exactly the package.

Current problems

Despite the theoretical advantages, IPWHL is a new project and thus has several problems

Lack of packages

A distribution doesn’t mean anything without packages, and IPWHL is indeed in need of them. As of the time I am writing this (2021-06-21), there are less than 100 packages declared in the database.

Introducing more packages would lead to maintenance problem: we cannot, as two sole maintainers, keep up with too many packages and make sure they’re all up-to-date. Therefore, please, come help us if you’re interested in this project.

Dependency Hell

Maybe you’ve heard of npm install is-even meme, if you hang out in some programming meme groups. It represents an underlying problem of having too many packages depending on each other. PyPI is saner, I would say, but it does have that problem.

xkcd comics "Dependency":
  A tower of blocks is shown. The upper half consists of many tiny blocks
  balanced on top of one another to form smaller towers, labeled:
  "All modern digital infrastructure"
  The blocks rest on larger blocks lower down in the image, finally on a
  single large block. This is balanced on top of a set of blocks on the left,
  and on the right, a single tiny block placed on its side. This one is
  labeled: A project some random person in Nebraska has been thanklessly
  maintaining since 2003
Original XKCD comics shared under a CC-BY-NC 2.5 License.
Transcript retrieved (with some edits) from ExplainXKCD shared under a CC-BY-SA 3.0 License.

I would even say if the package dependency were like the above illustration, it would be simple. In reality, circular dependency makes it impossible to declare one package without declaring the other, which can be demonstrated by this tensegrity shape:

a tensegrity structure
A tensegrity structure, drawn by me

Or, in some cases, such as for tox, it can even be like this:

A tensegrity icosahedron made from straws and string
Icosahedral tensegrity structure, retrieved from WikiMedia, authored by QuarterNotes, shared under a CC-BY-SA 4.0 License.

How to help

Rick from *Rick and Morty* dancing and singing 'I'm begging for
help'

Due to mentioned problems, it is critical for the project to have contribution. To start, please take a look at the manual page

Help declaring packages information

Currently, the packages with high priority is listed here:

https://todo.sr.ht/~cnx/ipwhl/5

Write docs

User and contribution manuals are not really clearly written, and some information are scattered across the mailing lists. You can help by compile them into a comprehensive structured manual.

Help resolving dependency conflicts

Maybe the real cheeses are the packages we helped along the way

Some packages cannot be in its latest version, due to some constraints.

A case in point is Sphinx, which depends on docutils and docutils-stubs. The latest version for docutils is 0.17.1, but so far the latest version for docutils-stubs depends on specifically 0.14 versions. Due to this conflicts, you can only install docutils 0.14.

Therefore, you can help developing and packaging docutils-stubs so we can have newer packages on IPWHL


Would you like to discuss this post? Email me!

Articles from blogs I read

Lithium and boron mine in Serbia

Rio Tinto wants to open a large lithium and boron mine in Serbia. That would be very useful for ren…

via Richard Stallman's Political Notes November 23, 2021

Online Discourse

I have been thinking for some time about writing some social commentary posts, but not falling into …

via Posts on Ariadna Vigo November 22, 2021

My desktop - November 2021

I decided it was time for a new “my desktop” post. It’s actually been one year since my latest post…

via Hund November 21, 2021
Generated by openring