[debian-mysql] Discussion on handling 5.6 related matters

[Akhil Mohan]

Hi Robie/ Everyone,

Thanks for your response which makes it easier to sort out the priority
list at this moment. As I get it, I am placing the tasks in order of
priority below:

 1. Upgrade Paths:
     1. Separation of data directory.
     2. mysql-defaults package should contain common functionality and
        own data directory as well.
     3. Handling upgrade scenarios from old format to new format that
        depends on mysql-defaults.
 2. Get remaining delta from 5.5 into 5.6 packaging source.
 3. Import latest released version of 5.6 into experimental and trusty
    universe.
 4. Get 5.6 as default in Debian testing and Ubuntu 14.10.

For now, as you said already, lets discuss on upgrade paths first.

You proposal looks promising but I suspect that users might find it
difficult to handle change of data directory location who wish continue
with MySQL. Moreover, MySQL 5.5 data dir can be directly taken over by
MySQL 5.6 using simple mysql_upgrade utility as it is being done at this
moment. There will no need for having separate /var/lib/mysql-5.6 data
dir for reason of binary incompatibility.

Another trouble that I see with forking the data dir is related to the
size of data dir. In case user has large data dir that goes into GBs or
even TBs then the copy would take noticeable amount of time. We might
need to show them a progress status/ bar and also handle situations like
copy of data dir ending midway with an error.

I would like to present a slight modification to the proposal with help
of controlled links. This would keep /var/lib/mysql intact owned by the
MySQL 5.x series keeping things simpler for all existing users who wish
to continue with MySQL. In case a user wants to install another fork of
MySQL then during installation fork.{pre,post}inst would detect existing
MySQL 5.6 and would ask if the user wishes to fork the data dir or start
fresh.

 1. Fresh start with blank data dir.
     1. Uninstall MySQL 5.x as regular removal.
     2. Create fork specific data dir /var/lib/fork or
        /var/lib/fork-version in case there can be compatibility issues.
     3. Move /var/lib/mysql to /var/lib/mysql.backup (snapshot)
     4. Create link /var/lib/mysql pointing to /var/lib/fork
     5. Install fork as regular installation.

The user can decide after installation to remove, backup or keep
/var/lib/mysql.backup. Fork would be able to access data dir directly at
/var/lib/fork. Same will remain accessible via /var/lib/mysql if needed
for administration purpose by user scripts.

 2. Fork the data dir.
     1. Uninstall MySQL 5.x as regular removal.
     2. Move /var/lib/mysql to /var/lib/fork and optionally copy
        /var/lib/fork to /var/lib/mysql.backup (snapshot)
     3. Create link /var/lib/mysql pointing to /var/lib/fork
     4. Install fork as regular installation.

Again, user can work on mysql.backup later if he decides so, otherwise
the default operations only move data dir that should be quick and more
manageable operation.

Please let me know your views on the modifications suggested above to
the initial proposal.

Regards,
Akhil

On Wednesday 02 July 2014 09:32 PM, Robie Basak wrote:
> Akhil,
>
> Sorry for the slow response.
>
> On Tue, Jun 17, 2014 at 11:21:25AM +0530, Akhil Mohan wrote:
>> In continuation to the discussion at UDS, it might be easier if we draft
>> the packaging design changes on mail to make sure everyone has the
>> correct perspective. I am listing down the TODO list having impact on
>> design of MySQL ecosystem in Debian and Ubuntu:
>>
>>  1. Taking forward 5.6 as default in Debian and Ubuntu.
> A work item for me is to examine the remaining delta for 5.5 in Ubuntu,
> and apply it to 5.6 as necessary, making sure to feed everything back to
> this list so that Debian can have the delta too.
>
>>  2. Separation of data directory and common packaging scripts as part of
>>     an independent package.
> I propose that when we're ready with things to go in there, we have a
> mysql-defaults source and binary package (probably "3.0 (native)"?).
> This could contain any commonality across variants as we work on them.
> Each variant that needs its functionality can depend on it.
>
> Maybe even (optionally, as needed) a Pre-Depends so that variants can
> use its scripts in preinst, etc? Or is this a bad idea?
>
>>  3. Giving each binary incompatible MySQL variant its own data directory
>>     at /var/lib/<fork>.
> I think this could be a first step. I propose that:
>
> 1) We deprecate /var/lib/mysql completely.
>
> 2) mysql-defaults takes over "ownership" of legacy /var/lib/mysql
> directories. I'm not sure if packages previously supplied shipped
> /var/lib/mysql from dpkg's perspective, but if so then mysql-defaults
> can Breaks/Replaces them.
>
> 3) mysql-defaults will "rm -rf /var/lib/mysql" on purge, but otherwise
> /var/lib/mysql should be treated as read-only by all other packages.
>
> 4) Each variant should have maintain its own directory in /var/lib.
> Whether that's (eg) /var/lib/mariadb-5.{5,6}/ or /var/lib/mariadb/ is
> up to the variants to decide. I imagine that this will depend on
> compatibility between versions. I can forsee the same sort of problem we
> have at the moment if multiple packages share a directory in /var/lib,
> so there should be a further restriction that only one binary package in
> the archive at one time may own one of these directories in /var/lib.
>
> 5) For automatic migration between variants where this is possible, a
> variant's binary package may _read_ from another variant's /var/lib
> directory. But it may not write or make any other changes.
>
> Example:
>
> * mysql-5.6 owns /var/lib/mysql-5.6
> * The user installs mariadb-10, which will own /var/lib/mariadb-10
> * mariadb-10.postinst detects /var/lib/mysql-5.6. It supports automatic
>   migration so it copies /var/lib/mariadb-10. The database state is now
>   forked.
>
> If the user later wants to switch back to mysql-5.6, he will either
> effectively go backwards in time, or he will need to migrate the data
> from /var/lib/mariadb-10 back to /var/lib/mysql-5.6. At this point he
> gets to choose whether to destroy the old data or to back it up first,
> etc.
>
> Common code for help with detecting what variants are in /var/lib, and
> perhaps to help the user with migration, could go into mysql-defaults.
>
>> Additionally I would like to know your view on adding MySQL 5.6.19 to
>> experimental and trusty as the latest released version. I can initiate
>> the work on 5.6.19 if you do not see any blockers.
> This sounds fine - though as 5.6 is in Debian experimental and Trusty
> universe, perhaps it might be an idea to sort out the /var/lib/mysql
> thing first, so that mysql-5.6 follows the new scheme from the start?
> I'm not sure though - since 5.6 is already in universe in Trusty, we
> probably do want to get the upgrade path right.
>
> Robie

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-mysql-maint/attachments/20140707/fe2483c6/attachment.html>