[Pkg-owncloud-maintainers] Upgrade report: From 7.0.12 to 8.0.10 and 8.2.2, files not decrypted anymore (was: Re: I am using distro packages and I tell you why
Martin Steigerwald
martin at lichtvoll.de
Wed Jan 6 16:18:18 UTC 2016
Am Montag, 4. Januar 2016, 09:27:54 CET schrieb David Prévot:
> Hi Martin,
Hi David,
> Thank you for (forwarding) your comments, it’s appreciated.
>
> Le 04/01/2016 07:58, Martin Steigerwald a écrit :
> > [David] already packaged newer Owncloud
> >
> > https://people.debian.org/~taffit/owncloud/
>
> JFTR, owncloud 8.2.2 is now available in experimental, and has been
> uploaded the same day as upstream (the Debian packages are usually
> published the same day as upstream by the way). The package in
> people.debian.org is currently published to provide an upgrade path for
> encryption users. From the latest NEWS entry:
>
> ++++++++++++++++++++++++++++++++++++
> Encryption (server-side) warning: do not upgrade from version 7 if you
> rely on this feature!
>
> The migration process documented upstream relies on upgrading first to
> version 8. A version 8 package is available from the following URL:
> <https://people.debian.org/~taffit/owncloud/>.
> If you use encryption, you should first upgrade to the version 8
> package, and then upgrade to this package and follow the upstream
> upgrading instruction from:
> <https://owncloud.org/blog/encryption-2-0-in-owncloud-server-8-1/>
> (in short, run under the proper credential (“sudo -u www-data”) the
> following commands: “occ app:enable encryption”,
> “occ encryption:enable”, and “occ encryption:migrate).
> ++++++++++++++++++++++++++++++++++++
> http://anonscm.debian.org/cgit/pkg-owncloud/owncloud.git/tree/debian/NEWS?h=
> stable8
>
> I hope we will be able to provide an easier path in time for Stretch,
> but in the mean time, this workaround seems to work.
This works somewhat okayish up to 8.0.10, but encryption breaks in various
inventive ways in 8.2.2 despite key migration being done with 8.0.10 before. I
am now once again back at 7.0.12 by restoring database and owncloud data files
from backup.
Also I think the price to pay on a Jessie server VM is to high. It pulls in a
lot of packages from unstable and experimental to update Owncloud.
It started with:
mondschein:~/owncloud> dpkg -i owncloud_8.0.10\~dfsg-1_all.deb
(Lese Datenbank ... 123433 Dateien und Verzeichnisse sind derzeit
installiert.)
Vorbereitung zum Entpacken von owncloud_8.0.10~dfsg-1_all.deb ...
Entpacken von owncloud (8.0.10~dfsg-1) über (7.0.12~dfsg-2) ...
dpkg: Abhängigkeitsprobleme verhindern Konfiguration von owncloud:
owncloud hängt ab von libjs-handlebars; aber:
Paket libjs-handlebars ist nicht installiert.
owncloud hängt ab von owncloud-doc (>= 8); aber:
Version von owncloud-doc auf dem System ist 0~20151214-1.
owncloud hängt ab von php-assetic (>= 1.3); aber:
Version von php-assetic auf dem System ist 1.2.0-2.
owncloud hängt ab von php-cssmin; aber:
Paket php-cssmin ist nicht installiert.
owncloud hängt ab von php-doctrine-dbal (>= 2.5.1-2~); aber:
Version von php-doctrine-dbal auf dem System ist 2.4.3-1.
owncloud hängt ab von php-getid3 (>= 1.9.9+dfsg-2~); aber:
Version von php-getid3 auf dem System ist 1.9.8-3.
owncloud hängt ab von php-opencloud (>= 1.13.0); aber:
Version von php-opencloud auf dem System ist 1.10.0-2.
owncloud hängt ab von php-patchwork-jsqueeze (>= 2.0.2); aber:
Paket php-patchwork-jsqueeze ist nicht installiert.
owncloud hängt ab von php-patchwork-utf8 (>= 1.2.1-2~); aber:
Version von php-patchwork-ut
dpkg: Fehler beim Bearbeiten des Paketes owncloud (--install):
Abhängigkeitsprobleme - verbleibt unkonfiguriert
Trigger für man-db (2.7.0.2-5) werden verarbeitet ...
Fehler traten auf beim Bearbeiten von:
owncloud
I installed this manually with apt-get download and dpkg -i:
mondschein:~/owncloud/deps-8.0.10> ls -1
libjs-handlebars_1.3.0-1_all.deb
owncloud-doc_8.2.2+dfsg-1_all.deb
php-assetic_1.3.2-1_all.deb
php-cssmin_3.0.4-1_all.deb
php-doctrine-annotations_1.2.7-1_all.deb
php-doctrine-cache_1.5.4-1_all.deb
php-doctrine-collections_1.3.0-2_all.deb
php-doctrine-common_2.4.3-1_all.deb
php-doctrine-common_2.6.1-1_all.deb
php-doctrine-dbal_2.4.5-1_all.deb
php-doctrine-dbal_2.5.4-1_all.deb
php-doctrine-inflector_1.1.0-1_all.deb
php-doctrine-lexer_1.0.1-3_all.deb
php-getid3_1.9.11+dfsg-1_all.deb
php-guzzle_3.9.3+dfsg-3_all.deb
php-json-patch_0.1.0-2_all.deb
php-opencloud_1.15.0+dfsg-1_all.deb
php-patchwork-jsqueeze_2.0.3-1_all.deb
php-patchwork-utf8_1.3.0-1_all.deb
php-pimple_1.1.1-1_all.deb
php-pimple_3.0.2-1_all.deb
php-psr-log_1.0.0-3_all.deb
php-punic_1.6.3-1_all.deb
php-randomlib_1.1.0-1_all.deb
php-sabre-dav_1.8.12-1_all.deb
php-sabre-vobject_2.1.7-1_all.deb
php-seclib_1.0.0-3_all.deb
php-securitylib_1.0.0-1_all.deb
php-symfony-console_2.7.7+dfsg-1_all.deb
php-symfony-event-dispatcher_2.7.7+dfsg-1_all.deb
php-symfony-process_2.7.7+dfsg-1_all.deb
php-symfony-routing_2.7.7+dfsg-1_all.deb
php-zipstreamer_0.7-1_all.deb
Then upgrade to 8.0.10 as follows:
Set log level to debug - current level: 'Warning'
Turned on maintenance mode
Checked database schema update
Checked database schema update for apps
Updated database
Updated <files_pdfviewer> to 0.7
Updated <activity> to 1.2.2
Updated <files_encryption> to 0.7.2
Updated <files_sharing> to 0.6.2
Updated <gallery> to 0.6.1
Turned off maintenance mode
Update successful
Reset log level to 'Warning'
I now tried the encryption migration as explained in your readme / referenced
blog entry:
mondschein:~#1> sudo -u www-data occ app:enable encryption
encryption not found
App was shown activated in web interface. So I left it with that. Decryption
of existing files worked anyway already. I continued as following:
mondschein:~> sudo -u www-data occ encryption:enable
[InvalidArgumentException]
Command "encryption:enable" is not defined.
Did you mean one of these?
app:enable
encryption:migrate-keys
I used the "migrate" method as mentioned in the blog entry. I think it
automatically selected "migrate-keys":
mondschein:~> sudo -u www-data occ encryption:migrate
Reorganize system folder structure
Migrating keys for users on backend Database
[a list of 7 users]
no errors.
So I thought I am fine. I was able to access the encrypted files just fine.
Thus I upgraded to 8.2.2, having to downgrade some PHP packages from
experimental to sid again, and pulling in more stuff from sid – maybe I
installed some packages in a two new version, I had experimental at 1 so it
didn´t pull automatically from there whats needed, thats why I used the apt-
get download / dpkg -i dance.
mondschein:~> LANG=C apt-get -t experimental install owncloud
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
owncloud : Depends: php-guzzlehttp (< 6) but 6.1.1-1 is to be installed
Depends: php-parser (< 2) but 2.0.0-1 is to be installed
Depends: php-sabre-dav-2.1 but it is not going to be installed
Depends: php-symfony-yaml (>= 2.6) but 2.3.21+dfsg-4+deb8u2 is to
be installed
Recommends: owncloud-apps (>= 0~8) but it is not going to be
installed
Recommends: php-aws-sdk (< 3) but it is not going to be installed
Recommends: php-aws-sdk (>= 2.8) but it is not going to be
installed
Recommends: php-dropbox but it is not going to be installed
Recommends: php-google-api-php-client (>= 1) but it is not going
to be installed
Recommends: php-smb but it is not going to be installed
Recommends: php5-apcu but it is not going to be installed or
php5-memcached but it is not going to be installed or
php5-redis but it is not going to be installed
Recommends: php5-imap but it is not going to be installed
Recommends: php5-ldap but it is not going to be installed
Recommends: smbclient but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
I solved these by:
apt-get install -t unstable php-guzzlehttp=5.3.0-1
apt-get install -t unstable php-parser=1.4.1-1
apt install -t unstable php-symfony-yaml
apt install php-sabre-dav-2.1 (from unstable)
I read that I need to install owncloud-apps and wanted to stop the upgrade o
to owncloud 8.2.2 after reading the NEWS entry via apt-listchanges. But it
stopped only after:
ownCloud or one of the apps require upgrade - only a limited number of
commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug - current level: 'Warning'
Turned on maintenance mode
Checking whether the database schema can be updated (this can take a long time
depending on the database size)
Checked database schema update
Checking updates of apps
Checking whether the database schema for <activity> can be updated (this can
take a long time depending on the database size)
Checking whether the database schema for <files_sharing> can be updated (this
can take a long time depending on the database size)
Checking whether the database schema for <files_trashbin> can be updated (this
can take a long time depending on the database size)
Checked database schema update for apps
Updating database schema
Updated database
Updating <files_texteditor> ...
Updated <files_texteditor> to 2.0
Updating <gallery> ...
Updated <gallery> to 14.2.0
Updating <files> ...
Updated <files> to 1.2.0
Updating <activity> ...
Updated <activity> to 2.1.3
Updating <files_sharing> ...
Updated <files_sharing> to 0.7.0
Updating <files_trashbin> ...
Updated <files_trashbin> to 0.7.0
Updating <files_versions> ...
Updated <files_versions> to 1.1.0
Update successful
Turned off maintenance mode
Reset log level to 'Warning'
Thus I made the apps install by casting a voodoo spell to make installing
owncloud-search work:
mondschein:~> LANG=C apt install owncloud-search
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
owncloud-search : Depends: php-zend-search but it is not going to be
installed
E: Unable to correct problems, you have held broken packages.
mondschein:~> LANG=C apt install -t unstable php-zend-search
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer
required:
php-guzzlehttp-promises php-guzzlehttp-psr7 php-psr-http-message
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
php-zend-hydrator php-zend-stdlib
Suggested packages:
php-zend-eventmanager php-zend-serializer php-zend-servicemanager php-zend-
filter
The following packages will be REMOVED:
zendframework
The following NEW packages will be installed:
php-zend-hydrator php-zend-search php-zend-stdlib
0 upgraded, 3 newly installed, 1 to remove and 585 not upgraded.
Need to get 201 kB of archives.
After this operation, 41.2 MB disk space will be freed.
Do you want to continue? [Y/n
mondschein:~> LANG=C apt-get -t experimental install --no-install-recommends
owncloud-apps
installed just fine there.
In webbrowser it allowed me to complete the upgrade of the apps. It was
successful.
But I was not able to access encrypted files. Web interface told me that
encryption keys still need migration, even tough I had an encryption-backup
folder in /srv/owncloud where I let owncloud store the data files.
I tried starting the migration from the web interface and got:
Es ist ein Problem aufgetreten, bitte überprüfe Deine Logdateien (Fehler:
Autoload path not allowed: /usr/share/owncloud/apps/encryption/lib/
migration.php)
(a problem occured, please view log files)
I also found this:
On this I thought it almost got it. Yet, I found:
mondschein:~#1> sudo -u www-data occ app:enable encryption
encryption enabled
mondschein:~#1> sudo -u www-data occ encryption:enable
Encryption is already enabled
No encryption module is loaded
mondschein:~> sudo -u www-data occ encryption:list-modules
mondschein:~>
So it didn´t see its encryption module. Maybe thats due to install owncloud-
apps later.
Okay, I went back to 8.0.10 and retried the migration as follows:
mondschein:~/owncloud> sudo -u www-data occ app:enable encryption
encryption is already enabled
mondschein:~/owncloud> sudo -u www-data occ encryption:enable
[InvalidArgumentException]
Command "encryption:enable" is not defined.
Did you mean one of these?
app:enable
encryption:migrate-keys
mondschein:~/owncloud#1> sudo -u www-data occ encryption:migrate-keys
Reorganize system folder structure
Migrating keys for users on backend Database
[same eight users, no errors reported
I was able to access the files just fine in 8.0.10 again.
I tried again upgrading to 8.2.2 this time owncloud-apps before owncloud, only
package that needs changing version between the both versions is php-seclib
between unstable and experimental versions.
This time again a complaint about migration of keys still needed, but I was
able to trigger it from the web interface this time. I even repeated it on
command line with migrate-keys method again without errors. Yet still an error
message in owncloud.log when trying to access a file, and file not decrypted:
{"reqId":"NfULHw
+mnNUWgc84xy77","remoteAddr":"188.174.194.205","app":"PHP","message":"Class OC
\\Files\\Stream\\Encryption contains 1 abstract method and must therefore be
declared abstract or implement the remaining methods (Icewind\\Streams\
\Directory::dir_opendir) at \/usr\/share\/owncloud\/lib\/private\/files\/
stream\/encryption.php#30","level":3,"time":"2016-01-06T14:55:05+00:00"}
Yet this time encryption module seems to be set properly:
So I went back to 8.0.10 again and back up to 8.2.2 and this time I got:
[15:56:12] <helios21> mondschein:~#130> sudo -u www-data occ encryption:list-
modules
[15:56:12] <helios21> - OC_DEFAULT_MODULE: Default encryption module
[default*]
[15:56:18] <helios21> und last attempt this was empty.
[15:57:30] <helios21> okay, yes, I have a new tab in admin settings now,
server side encryption. It is activated and the default encryption module is
selected
(copied from IRC channel where I wrote to)
As I wanted back a running system soon and didn´t know what to do, I retried
the going back to 8.0.10 again, but this time I was not even able to access
the files in 8.0.10 anymore.
For the sake of it I upgraded to 8.2.2 once again, yet no luck.
But this time even different error messages:
{"reqId":"nR+QCHYVMTKAX7eszICj","remoteAddr":"","app":"no app in
context","message":"did not move key \"skel.shareKey\" could not find the
corresponding file in \/data\/skel\/files.Most likely the key was already
moved in a previous migration run and is already on the right place.","level":
2,"time":"2016-01-06T15:20:43+00:00"}
{"reqId":"nR+QCHYVMTKAX7eszICj","remoteAddr":"","app":"no app in
context","message":"did not move key \"fileKey\" could not find the
corresponding file in \/data\/skel\/files.Most likely the key was already
moved in a previous migration run and is already on the right place.","level":
2,"time":"2016-01-06T15:20:43+00:00"}
{"reqId":"nR+QCHYVMTKAX7eszICj","remoteAddr":"","app":"no app in
context","message":"did not move key \"skel.shareKey\" could not find the
corresponding file in \/data\/skel\/files.Most likely the key was already
moved in a previous migration run and is already on the right place.","level":
2,"time":"2016-01-06T15:20:43+00:00"}
{"reqId":"nR+QCHYVMTKAX7eszICj","remoteAddr":"","app":"no app in
context","message":"did not move key \"fileKey\" could not find the
corresponding file in \/data\/skel\/files.Most likely the key was already
moved in a previous migration run and is already on the right place.","level":
2,"time":"2016-01-06T15:20:43+00:00"}
{"reqId":"nR+QCHYVMTKAX7eszICj","remoteAddr":"","app":"no app in
context","message":"did not move key \"skel.shareKey\" could not find the
corresponding file in \/data\/skel\/files.Most likely the key was already
moved in a previous migration run and is already on the right place.","level":
2,"time":"2016-01-06T15:20:43+00:00"}
{"reqId":"nR+QCHYVMTKAX7eszICj","remoteAddr":"","app":"no app in
context","message":"did not move key \"fileKey\" could not find the
corresponding file in \/data\/skel\/files.Most likely the key was already
moved in a previous migration run and is already on the right place.","level":
2,"time":"2016-01-06T15:20:43+00:00"}
{"reqId":"nR+QCHYVMTKAX7eszICj","remoteAddr":"","app":"no app in
context","message":"did not move key \"skel.shareKey\" could not find the
corresponding file in \/data\/skel\/files.Most likely the key was already
moved in a previous migration run and is already on the right place.","level":
2,"time":"2016-01-06T15:20:43+00:00"}
{"reqId":"GFKoqc7ZWEpzJtDCms6b","remoteAddr":"188.174.194.205","app":"no app
in context","message":"Exception: {\"Exception\":\"OCA\\\\Encryption\\\
\Exceptions\\\\MultiKeyDecryptException\",\"Message\":\"multikeydecrypt with
share key failed:error:0407109F:rsa
routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error\",\"Code\":0,
\"Trace\":\"#0 \\\/usr\\\/share\\\/owncloud\\\/apps\\\/encryption\\\/lib\\\/
keymanager.php(406): OCA\\\\Encryption\\\\Crypto\\\\Crypt->multiKeyDecrypt('qw
\\\\xA3\\\\xAF\\\\xB0\\\\x13\\\\xF6\\\\xBF\\\\x94\\\\xE7\\\\x91\\\\xEAy\\\\xAA
\\\\xDD...', '6I\\\\x90\\\\xE1\\\\x7F\\\\xC1$8\\\\xFBtY\\\\v\\\\xB0\\\\x1A\\\
\x02...', '-----BEGIN PRIV...')\\n#1 \\\/usr\\\/share\\\/owncloud\\\/apps\\\/
encryption\\\/lib\\\/crypto\\\/encryption.php(419): OCA\\\\Encryption\\\
\KeyManager->getFileKey('\\\/kyradon\\\/files\\\/...', 'kyradon')\\n#2 \\\/usr
\\\/share\\\/owncloud\\\/lib\\\/private\\\/files\\\/storage\\\/wrapper\\\/
encryption.php(289): OCA\\\\Encryption\\\\Crypto\\\\Encryption->isReadable('\\
\/kyradon\\\/files\\\/...', 'kyradon')\\n#3 \\\/usr\\\/share\\\/owncloud\\\/
lib\\\/private\\\/files\\\/storage\\\/wrapper\\\/wrapper.php(161): OC\\\\Files
\\\\Storage\\\\Wrapper\\\\Encryption->isReadable('files\\\/ownCloudU...')\\n#4
\\\/usr\\\/share\\\/owncloud\\\/lib\\\/private\\\/files\\\/view.php(1023): OC\
\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->isReadable('files\\\/ownCloudU...')
\\n#5 \\\/usr\\\/share\\\/owncloud\\\/lib\\\/private\\\/files\\\/
view.php(399): OC\\\\Files\\\\View->basicOperation('isReadable', '\\\/\\\/
ownCloudUserM...')\\n#6 \\\/usr\\\/share\\\/owncloud\\\/lib\\\/private\\\/
files\\\/filesystem.php(672): OC\\\\Files\\\\View->isReadable('\\\/\\\/
ownCloudUserM...')\\n#7 \\\/usr\\\/share\\\/owncloud\\\/lib\\\/private\\\/
files.php(164): OC\\\\Files\\\\Filesystem::isReadable('\\\/\\\/
ownCloudUserM...')\\n#8 \\\/usr\\\/share\\\/owncloud\\\/lib\\\/private\\\/
files.php(90): OC_Files::getSingleFile(Object(OC\\\\Files\\\\View), '\\\/',
'ownCloudUserMan...', false)\\n#9 \\\/usr\\\/share\\\/owncloud\\\/apps\\\/
files\\\/ajax\\\/download.php(53): OC_Files::get('\\\/', Array, false)\\n#10 \
\\/usr\\\/share\\\/owncloud\\\/lib\\\/private\\\/route\\\/route.php(154) :
runtime-created function(1): require_once('\\\/usr\\\/share\\\/ownc...')\\n#11
[internal function]: __lambda_func(Array)\\n#12 \\\/usr\\\/share\\\/owncloud\\
\/lib\\\/private\\\/route\\\/router.php(291): call_user_func('\\\
\x00lambda_626', Array)\\n#13 \\\/usr\\\/share\\\/owncloud\\\/lib\\\/
base.php(851): OC\\\\Route\\\\Router->match('\\\/apps\\\/files\\\/aja...')\
\n#14 \\\/usr\\\/share\\\/owncloud\\\/index.php(39): OC::handleRequest()\\n#15
{main}\",\"File\":\"\\\/usr\\\/share\\\/owncloud\\\/apps\\\/encryption\\\/lib\
\\/crypto\\\/crypt.php\",\"Line\":563}","level":
3,"time":"2016-01-06T15:21:03+00:00"}
Unfortunately I do not have complete logs anymore, cause:
mondschein:~> ls -l /var/log/owncloud.log*
-rw-r----- 1 www-data adm 227 Jan 6 16:34 /var/log/owncloud.log
maybe due to me purging the owncloud package completely and then all further
packages from unstable and experimental to go back to 7.0.12 without to much
downgrading hassle.
I do have the broken 8.2.2 data file directory in case you have any questions
about its contents. It contains a backup directory for each of my key
migration attempts:
mondschein:/srv/backup/owncloud-8.2.2-upgrade-broke> ls -ld
encryption_migration_backup_2016-01-06_1*
drwxr-sr-x 1 www-data www-data 94 Jan 6 14:50
encryption_migration_backup_2016-01-06_13-50-08
drwxr-sr-x 1 www-data www-data 32 Jan 6 14:56
encryption_migration_backup_2016-01-06_13-56-55
drwxr-sr-x 1 www-data www-data 32 Jan 6 15:43
encryption_migration_backup_2016-01-06_14-43-11
drwxr-sr-x 1 www-data www-data 32 Jan 6 15:51
encryption_migration_backup_2016-01-06_14-51-03
drwxr-sr-x 1 www-data www-data 32 Jan 6 15:52
encryption_migration_backup_2016-01-06_14-52-58
drwxr-sr-x 1 www-data www-data 32 Jan 6 16:20
encryption_migration_backup_2016-01-06_15-20-06
I am now safely back at owncloud 7.0.12 from unstable and wonder whether
upgrading to 8.2.2 within a Jessie system makes sense considering the amount
of packages from unstable and experimental needed. But even with Stretch I
think upgrading an owncloud instance with server side encryption out to be
easier than my todays experience.
I tried to keep good log of what I was doing when, but there is a non-zero
chance that I might have mixed up something in here. But I think its basically
correct.
Ciao,
--
Martin
More information about the Pkg-owncloud-maintainers
mailing list