[Pkg-owncloud-maintainers] Upstream making our life more and more painful (Was: [owncloud-devel] Bug#815963: Warn users about unsupported upgrade path)

[Martin Steigerwald]

Added Owncloud upstream packaging team into Cc.

On Sonntag, 28. Februar 2016 03:54:26 CET Jos Poortvliet wrote:
> replying on phone. blame faulty text completion/correction for any rudeness!
> On Feb 27, 2016 4:39 PM, "Martin Steigerwald" <martin at lichtvoll.de> wrote:
> > On Samstag, 27. Februar 2016 15:01:57 CET David Prévot wrote:
> > > Hi,
> > 
> > Hello David,
> > 
> > > Again, upstream ownCloud developers showing how badly cooperation can
> 
> be…
> 
> > > Le 27/02/2016 10:49, Lukas Reschke a écrit :
> > > > This is super dangerous stuff from Debian and I *HIGHLY* would advise
> > > > anybody from NOT using distribution packages for ownCloud.
> > > > 
> > > > Highly irresponsible from them to risk user data like that. Honestly,
> 
> the
> 
> > > > maintainer should know better.
> > > 
> > > They are becoming more and more vocal about so called issues, would that
> > > be in their mailing lists, in their own personal blog posts, in their
> > > bug tracker… The ambiance becomes more and more unbearable.
> > 
> > *hugs*
> > 
> > I more and more wonder about an alternative to Owncloud.
> > 
> > One with a development model where versions are also deploy- and
> 
> supportable
> 
> > without updating a docker container directly from git repos.
> 
> Note that we aren't happy with the upgrade process either. Our goal has
> always been to make upgrading entirely automatic and fast - but we value
> user data more than ease or speed so right now, it isn't like that.
> OwnCloud 9.0 introduces a rewritten, stand-alone updater which is a first
> step in the direction of fully unattended, safe and fast upgrades. Note
> that the new upgrade routine will get its first run when people upgrade
> from 9.0.0 to 9.0.1, it isn't used (afaik) when upgrading from 8.2.
> 
> With this new updater, some day it might become safe to skip versions on
> upgrade. Right now, our full time engineering team thinks they can't do it
> with the current architecture within reasonable time and effort. That it

While on one hand I have sympathy with not supporting skipping upgrades as 
actually Debian does not either, on the other hand, there is also a huge 
difference in upgrade frequency. A Debian upgrade is about all three years, 
with lots of time to plan for it and even the option to stick with LTS support 
for some time. With Owncloud the upgrade frequency is much higher.

That said I develop for a Ruby on Rails app at my employer here and then and 
there I learned that it is a default feature to be able to skip database 
migrations. There each database migration step has a time stamp. And its 
totally safe to have more than one db migration between deployments. It will 
just check what migrations it did already done and then apply the rest. This 
is even deeply rooted in the actual framework, the app developer doesn´t have 
to care about it.

That is of course just the db migration, in addition to this Owncloud 
developers did lots of changes in encryption and its mostly a challenge to get 
this right for the upgrade of Owncloud as packaged in Debian. I tried it twice 
meanwhile, both times it was a disaster, last time it could have worked, but I 
misunderstood some of the instructions David wrote in the README.Debian file 
and he clarified it afterwards.

That said, upstream learned and I bet upstream may not into totally changing 
encryption every year or so, but more into finding a robust model that works 
for some time to go. If Owncloud matures, and I bet and sure hope it does, 
then maybe disruptive changes like these become more rare.

> why we kindly ask you not to try, due to the risks for users' data. Or, if
> you really need this, talk to us - I would be happy to organize a call or
> IRC session about how to help people on 7.0.x to a newer release in the
> smoothest and safest way. We have a bit of experience with upgrading
> ownCloud instances...

I thank you for this offer. It is up for the actual packagers in Debian like 
David and Sandro (for the client) to decide about it. I think it is a step in 
a good direction.

David wrote in the mail I replied to that he is shortly before a good upgrade 
path for Debian Jessie to Stretch. I am not sure how it works out. We also 
have at least one more tester – than me – who has Owncloud as packaged in 
Debian and from Upstream willing to test upgrade scenarios. He claimed to have 
faced data loss even with upstream packaging as he has one VM using upstream 
packaging.

> > You have my respect for your work and I greatly value it.
> 
> Not all ownCloud contributors might always be nice about it but we really
> do appreciate the efforts. Distributions provide an important service to
> the Foss ecosystem in general and bring ownCloud to users - that is a good
> thing.
> 
> I would like to invite all of you to the ownCloud conference in September -
> it would be nice to have a sit down and see how we can improve things; and
> the work together to make it happen. I would be happy to arrange travel
> support for those who need it.

Thank you for this offer as well.

I do think there are certain topics that can be discussed:

1) In general how to bring meetings needs of Debian and upstream into a 
balance that can work for both.

2) Also how to resolve current bitterness, resentment and frustrations on both 
sides between upstream and Debian. I do think this is perfectly possible, but 
it may be good to meet in person for this. I found a huge difference between 
discussing things on the net and meeting Debian maintainers in person at 
DebConf 2015 in Heidelberg. Even with David himself. I had some mail exchanges 
with him before meeting him at DebConf 2015 and thought "Who that dude isn´t 
easy to handle", just to learn that he is so friendly and easy to talk with at 
DebConf. Meeting in person does make a difference. I did not only experience 
this with David, but with other Debian maintainers I thought I knew from the 
net.

3) Is there a way to keep most dependencies somewhat stable long enough for 
Owncloud to make providing Owncloud backports feasible for at least some time. 
I actually do not know how upstream packaging handles this, do you stuff every 
dependency that is not available in Debian stable into the package itself? 
This, like running Owncloud in a docker, easily doubles the effort needed to 
provide security support for it. Whenever there is a security update for a 
dependencies, an update of the owncloud packaging would be required, while 
when relying mostly on Debian stable packages for dependencies shifts the 
responsibility for that to the Debian security team.

4) Also an option: How to make upstream packagaging and Debian packaging 
compatible enough so that users can switch in between them.


These are at least some of the topics I can come up with without even thinking 
much about it. David might have different topics and you may have different 
topics as well such as:

5) How to make sure Debian stays up to date enough or has all necessary 
security updates. I do personally think David does a good shop at backporting 
security fixes, whether that is enough or not, I don´t know, so far I am quite 
confident that nobody broke into my internet facing Owncloud setup on Debian 
Jessie VM. I think its important to focus on facts there. I.e. are there any 
concrete, important security updates missing from current Debian packaging? 
Spreading FUD about the Debian packaging is not going to help IMO.

6) How to ensure smooth upgrade pathes that do not have people who use the 
packaging in Debian run to your support channels – I do think there is a 
shared interest as David works hard to make sure the upgrade path is smooth, I 
bet you can imagine he isn´t interested in data loss either, who is, actually? 
And there are at least some testers will to help testing it –  …

7) … or even how to inform Debian users to use Debian support channels about 
at least some of the issues they are facing. That said, you did already do a 
step into that direction and I ask, do you really receive lots of support 
requests from people using the packages in Debian?

That was just some guess work, but from what I read from your mails I do think 
these could be some topics you are interested in.


I do think basically both Debian maintainers and Upstream developers and 
packagers act with good intentions, yet with different ideas about release 
schedule, packaging and upgrade handling. So I do believe there is room for 
improving the situation. It may need to agree to disagree in some areas and 
work from there tough. What you do, makes sense for you, what Debian 
maintainers do, make sense for them. In respect for this I think its easier to 
move forward and find solutions that can work for everyone involved.

I do not speak for the actual Debian maintainers but as an user and someone 
who is willing to help test upgrade with encryption, I´d really like to see 
and contribute progress in this area. I would be sad if Owncloud would be 
removed from Debian repositories in the future. Especially after the huge 
efforts mostly David spent to make it work smoothly.

I am even willing to moderate, yet, I bet I might be too biased already and 
its better to have someone more neutral for moderation.

Thank you,
-- 
Martin