[Blend-tinker-devel] wiki.d.o on a git-backed engine

Tue Jan 14 11:55:13 GMT 2025

[moved from thread at debian-devel at lists.debian.org]

Quoting Serafeim (Serafi) Zanikolas (2025-01-13 22:06:01)
> On Mon Jan 13, 2025 at 12:45 AM CET, Jonas Smedegaard wrote:
> > Quoting Serafeim (Serafi) Zanikolas (2025-01-12 21:54:58)
> > > what would be truly amazing, imho, would be the whole wiki on git. that'd allow
> > > for mass-updates, and reusing one's code (salsa) workflows for documentation
> >
> > For those not only wishing that others made that happen, but consider
> > scratching that particular itch themselves, I am happy to chat with you
> > more detailed about what that involves, and how it is a quite exciting
> > challenge - so exciting, in fact, that Debian has had several attempts
> > at doing a migration over the years, but none so far have succeeded.
> [..]
> > But let's not hijack any further this thread about welcoming newcomers,
>
> let's create a new thread then :)
>
> > I just wanted to chime in to encourage anyone wanting to do the wiki
> > modernization to get in touch with me to discuss further.
>
> thank you for the offer but why not have the follow up in a publicly archived
> list? happy to switch to -www, if -devel is not ideal
>
> I think that a retrospective/postmortem of past attempts would be very valuable.
> I offer to write one if you'd be okay to share all of the context with me (you
> could review it before it gets published).
>
> also, I'd think that nailing down the requirements for a new platform and for
> the content to be migrated (e.g. drop any pages that are >X years old) would be
> an important prerequisite for any technical work. has this been done in previous
> migration attempts? to start with, do we have "rough consensus" that a
> git-backed wiki would be preferable?

# Wiki

I guess we can agree that we want to continue to have a [quick-web],
i.e. a collaborative collection of concept pages quickly user-editable
and revision-controlled.

I mention this because wiki has been used more broadly for systems less
collaborative or less user-oriented or not revision-controlled.

Do we agree that the above are requirements?

Even if we think we agree, let's revisit them later, as they might turn
out to need some wiggle room, e.g. making it less easy and less quick
for users to edit to avoid spam.

# User interface(s)

One detail we might disagree on is the notion of "user", which is tied
to the notion of "web": The originial WikiWikiWeb was edited via a web
*browser* whereas some newer systems are edited offline and then pushed
(via web or other protocols) to be read via a web browser. A few of
those systems offer web-based editing as well, as an optional addon.

Strictly requiring web-browser-based editing limits available options.

Personally I am leaning towards separation of concerns, in that I want
flexibility of choice of as fast and lightweight as possible static site
generator (see below) with geeky-user editing via a git interface as
required and simple-user editing via web interface as an optional addon.
I envision approaching that separation through embracing the standard
Micropub, and am slowly looking into implementing a [proxy] service to
address that.  I am lousy at coding, however, so if you think that
sounds exciting then please don't wait for me to solve that challenge
but consider joining me!

[quick-web]: https://en.wikipedia.org/wiki/Wiki#History

[Micropub]: https://www.w3.org/TR/micropub/

[proxy]: https://salsa.debian.org/tinker-team/website-edit

# Markup

Some esp. early wikis use markup now considered obscure, where most
newer systems use some flavor of Markdown.

Markdown-based wikis sometimes deviate further through custom markers,
and some add metadata at the top, most commonly as YAML or TOML data.

Do we want to require a certain markup, or leave that to whatever system
we want for different reasons?

This obviously affects the migration challenge, but perhaps not decide
now, but just keep it in mind when considering system options.

# Backend

Most wikis store page content as files in a POSIX filesystem, or as
strings in an SQL database, or as documents in a NoSQL database, as is
standard for those backends, with revision control and web editing added
as custom logic on top of that.

Git-based wikis use a standardized revision control, with web access as
custom logic on top.

The use of a standardised version control system is a key feature for
me, and judging from the topic of this thread also for others.

Requiring git-based backend limits options, and perhaps most notably
excludes the undeniably popular system Mediawiki, used e.g. for
Wikipedia. Some disagree with me on that, though:
https://webmasters.stackexchange.com/a/82433

# Rendering

Some wikis render each page on-demand, and some render all pages ahead
of time after each content edit. The latter are called static web
compilers.

I am a big fan of static web compilers, but whether they are more
efficient obviously depends on how frequently a site is being edited
versus viewed, and how expensive a full site recompilation is.

# Git-backed options with web-editor

A nice but potentially dated overview is here:
https://paulhammant.com/2017/09/23/wikis-that-use-source-control-for-their-backing-store/

# Git-backed options without web-editor

Lots of options, since most static web compilers can be scripted to use
git as backend. Most popular is probably are Hugo and Jekyll.

My personal favorite is Zola, less popular but like hugo among the
fastest and unlike hugo uses a more common template logic.  I am also
pleased by Zola being written in Rust where Hugo is on go.  Hugo is
however already available in Debian, Zola is not (I am slowly working on
it).

# Migration

MoinMoin use a filesystem-based backend with each page revision in a
sequence-numbered subdirectory. Content markup predates Markdown and is
called Creole. A plugin mechanism introduces custom markers that need
caring for as well.

Around 2005 I packaged the Perl modules usable for transforming wiki markup,
including from MoinMoin Creole to flavors of Markdown.

An attempt to migrate to Ikiwiki was investigated around 2008-2009.  I
vaguely remembered that effort to be lead by Frank Lin PIAT, but reading
some archived emails it seems I was more convinced than him on that, so
I am no longer sure what happened back then, and it was before I gained
familiarity with git so I have no code lying around related to that.

The best that I know of is the work by anarcat and others, maintained at
https://ikiwiki.info/tips/convert_moinmoin_to_ikiwiki/

I have some unfinished code converting a relatively large ikiwiki site
to Zola, which might have some use also for migrating to other systems
like hugo.

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/
 * Sponsorship: https://ko-fi.com/drjones

 [x] quote me freely  [ ] ask before reusing  [ ] keep private
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 931 bytes
Desc: signature
URL: <http://alioth-lists.debian.net/pipermail/blend-tinker-devel/attachments/20250114/ec5953d0/attachment.sig>