[Soc-coordination] Final summary of ZFS on Linux integration project

Thu Sep 26 06:52:20 UTC 2013

On Thu, Sep 26, 2013 at 1:54 PM, Darik Horn <dajhorn at vanadac.com> wrote:
> On Tue, Sep 24, 2013 at 12:28 AM, Aron Xu <happyaron.xu at gmail.com> wrote:
>> Hi everyone,
>>
>> This is the final summary of ZFS on Linux integration project.
>>
>> Looking at the whole project, the overall goal is accomplished with
>> direct and indirect help from different parties - ZoL project, D-I
>> community, and of course my mentor.
>
> Where are the final products posted?  I audited your work, and I can't
> find a complete deliverable for any of the stated GSoC project goals.
>

Have you read _any_ of my reports? For example deliverable, see:
http://people.debian.org/~aron/gsoc2013/week11/

I don't understand why you would like to be challenging here and there
just because you didn't get a DD identity and your dreamed fame for
the work you have done. This will be my only response to this round of
your replies, as both my mentor and me have said that you were
starting non-sense threads with blaming and even sometimes without
respect at the very beginning of Debian's ZoL related work.

Reference for everyone else:
 http://lists.alioth.debian.org/pipermail/pkg-zfsonlinux-devel/2013-May/000023.html
 http://lists.alioth.debian.org/pipermail/pkg-zfsonlinux-devel/2013-May/thread.html

> Now quoting from the specification at:
> https://wiki.debian.org/SummerOfCode2013/Projects#ZFS_on_Linux_integration
>
>
>> 1. Integrated ZFS on Linux support via "apt-get install" and support for ZFS root filesystem.
>
> In your last status update, you said that only packages for non-root
> installation are ready, but surely you have partial work to submit.
> Where are the preliminary patches for grub, util-linux,
> debian-installer, etc, that would support a ZFS root filesystem on
> Debian?
>

1) debian-installer ones means partman-zfs? It's landed, and I'd thank
Turbo Fredriksson very much for his work which gave me a new way to
go. See it's already build on kfreebsd-any and linux-any at this link:
http://packages.qa.debian.org/p/partman-zfs.html
2) grub2:
 1st attempt (failed): http://people.debian.org/~aron/gsoc2013/week10/
 2nd attempt (success, thanks the help from ryao):
http://people.debian.org/~aron/gsoc2013/week11/grub2/
 Also I've confirmed with upstream for several times and finally he
pushed it into bzr, not anytime near your contact.
3) util-linux: not very needed so far.
4) dkms: not needing your patches, upstream confirmed their preference
to a priority solution than a dependency solution.

I understand you find that you need to modify lots of stuff to make it
somewhat "better", but you are not patient and rarely discuss where is
the best but just write patches to hack, that's the reason your work
cannot be directly used in any of Debian/Ubuntu.

>
>> 2. Write or improve wrappers including zfs-auto-snapshot, apt-zfs-snapshot, and maybe more.
>
> I haven't received any submissions for zfs-auto-snapshot upstream, but
> apt-zfs-snapshot is interesting.  Where can I download it?
>

Your work on zfs-auto-snapshot is great, and I don't see the need of
submitting my debian/ stuff upstream, no? I've written in the report
that d-i support used more time than expected, so apt-zfs-snapshot and
beadm weren't worked on. But as one of the starters of pkg-zfsonlinux,
I will definitely work on them after this SoC.

>
>> 3. Improve debian-installer/partman integration. This involves creating volumes when doing the initial installation, and maybe separate system directories (/boot, /usr, /lib, etc) from user directories (e.g. /home).
>
> You can't fulfill this goal before the deadline due to externalities
> involving licensing and system policy, but you were aware of these
> risks before starting the job.

You are wrong, as agreed in #zfsonlinux, the goal is to make it easy
enough to get it in d-i, even the binary modules can't be included in
official archive. So in the link above I've made a working image for
people to try (limited testing applies, strongly discouraged  for
production use).

> Additionally, the stuff that you did do seems to be a repack of the
> existing kFreeBSD support.  I don't see anything new for ZoL in the
> debian-installer repository, and your contributions to partman-zfs
> don't fix bugs like #721578, which could be big enough for a separate
> project.
>

The current code is largely based on LVM code, and you agreed it could
be big enough for another project. Actually it needs very intrusive
changes and even re-write.

>
>> 4. A beadm-like tool. beadm is a wrapper available on Solaris, and it is dependent to previous item like the initial filesystem layout created by the installer. Changes to debian-installer/partman and grub2 would be required.
>
> The beadm code from Solaris doesn't compile cleanly on Debian and it
> doesn't recognize the Linux loader configuration.  Where can I get the
> ported version?
>
>
>> 5. Optionally a testing tool to verify if the integration works with common setups, i.e. automation testing using virtual machines to simulate common setups and usages (desktops, servers, storage servers, etc).
>
> Did any work happen here?  Where is the test matrix and status page?
> Did you post virtual machines for common setups?  Did you engage the
> user community for deployment scenarios?
>

See above.

>
>> What the student will learn: experience of filesystem integration and collaboration with different teams
>
> This bullet is an implied sixth goal, but you have zero participation
> in the upstream lists, zero participation in the upstream issue
> tracker, and you failed to publicize your work or otherwise make it
> easily accessible.
>

It only means you always failed to get information you are eager to
know in time, just like the thread you started when this GSoC is
nearly started and you said you don't know that before. No one has the
responsibility of informing everything just because if you don't you
would blaming. I've been hanging around upstream IRC for quite some
time, and have many conversations publicly or privately with people
that can help me, or want to know the progress.

>
> Furthermore, the project specification is somewhat inconsistent with
> the project proposal.  Now quoting from:
> https://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/aron/12001
>
>
>> 1. “apt-get”-able zfs support using DKMS
>
> This work was finished years ago -- certainly in time for the ITP
> ticket opened in September 2012 -- so you shouldn't get any credit for
> it.
>
> Nevertheless, I ran a code analysis on your packaging repository.  In
> total, you have 31 commits in the SPL overlay, 69 commits in the ZFS
> overlay, and 0 commits upstream.  And the Alioth repository is empty,
> which is the landing page for new users.
>
> However, excluding upstream merges and boilerplate changes, there are
> only 13 functional changes in SPL and 34 functional changes in ZFS,
> most of which are trivial.  Some of these are duds like the unmatched
> parenthesis that should have been caught in testing or review.
>
> Objectively the diffstat is tiny, and subjectively you've personally
> written nearly zero actual useful code (that is publicly available).
>

Your dkms patches isn't used at all, and your decisions in zfs-dkms is
not really satisfactory. I did test most of the possibilities that you
explained in comments or public mailing list archive for the weird
lines, and the corrected behavior cannot be measured only by lines of
changes, but how they affect the behavior and the analysis work done
behind, that used up lots of time of my mentor and me starting from
the preparation stage of this GSoC. I'd like to reiterate that I
understand you make those changes for a reason, but I have been trying
out different software and hardware configuration I can achieve and
see if they are really matters/hurts and to remove any bits that could
be improper for a distribution that don't have a tight user community
like upstream that you can here feedback very quickly.

>
>> 2. “apt-get”-able tools like apt-zfs-snapshot, zfs-auto-snapshot, and beadm-like tool.
>> 3. Debian Installer support through a pre-build debian-installer image with zfs support included.
>> 4. Make Debian Installer create root file system using transparent volumes by creating FHS directories as independent dataset, and tweak the default volume setting for which system is to be used.
>> 5. The beadm-like tool, is a wrapper program to help users make snapshot properly, switch between different set of snapshots by changing default volume of the file system and make changes to Grub2 configurations when needed.
>> 6. Similar to the beadm-like tool, other tools like apt-zfs-snapshot can make use of the dataset layout, so that users can get similar experience like switching between systems without affecting user directories.
>
> Same as above.  Where is the code posted?
>
> Even if you get goal #1 for free and goal #3 for effort, then your
> current score is still 2/6 for failing to deliver root capabilities,
> the apt-zfs-snapshot and beadm utilities, a testing tool or framework,
> and anything else that is not plain packaging.
>
> --
> Darik Horn <dajhorn at vanadac.com>

See? But nevertheless I'm tired of replying non-sense stuff, I still
have patches pending in mailing list to look at and commit that people
already pinged me twice.

-- 
Regards,
Aron Xu