[Android-tools-devel] Updating android-platform-system-core

[Hans-Christoph Steiner]

Sounds good.  Three small things

* use the 5.1.1+r8 version format for the source package since it is a closer
match to the upstream version

* use 22 instead of 22.0.0 for android-platform-tools to match the upstream
version.  That won't prevent a 22.1 if upstream decides to do such a thing,
but so far they have only done major version releases of Platform-tools.

* the metapackage does not need to have the same name as the source package,
so use android-sdk-tools and not androidsdk-tools.  Then when it is easy, the
source package can be renamed to android-sdk-tools, but that is low priority.

.hc

殷啟聰:
> Hi Hans,
> 
> This is my overall plan. Assume that the latest release of Android is
> android-5.1.1_r8:
> 
> For every source package of Android repositories, we use 5.1.1.8 as the
> source version, so every executable and library of Android SDK will be
> versioned 5.1.1.8.
> 
> Then we have another source package "android-platform-development" based on
> platform/development which produces binary packages like androidsdk-common
> (or separated as androidsdk-platform-tools-common and so on) that install
> miscellaneous files belonging to each toolset. In <
> https://android.googlesource.com/platform/development/+/master/build/sdk.atree>
> we can see where to find the files to install for every SDK toolset.
> 
> Then we have three meta binary packages: androidsdk-platform-tools,
> android-sdk-build-tools and androidsdk-tools (which is why I suggested
> rename the existing androidsdk-tools), each of which depends on the tools
> and common files packages belonging to them. This three meta packages have
> their own versions following the original SDK version scheme. Thus
> androidsdk-build-tools can be versioned 22.1.3, androidsdk-platform-tools
> can be versioned 22.0.0 and androidsdk-tools can be versioned 24.2.3. We
> can make their depends versioned, for example
> androidsdk-platform-tools_22.0.0 depends on adb (>= 5.1.1.8), and maybe
> androidsdk-platform-tools_23.0.0 depends on adb (>= 6.0.0.1).
> 
> As of my observation, the minor part of the versions of androidsdk-tools
> and libgradle-android-java is the same, so I guess there will be no problem
> for them.
> 
> Cheers,
> Kai-Chung Yan
> 
> Hans-Christoph Steiner <hans at at.or.at> 於 2015年7月21日週二 上午6:17寫道：
> 
>>
>> Got some useful advice from #debian-devel: binary packages can have
>> separate
>> versions from the source package using dpkg-gencontrol -v
>>
>> So in this case we can use the tag version for the source package in the
>> debian/changelog, like this: 5.1.1+r7 (letters are allowed in versions).
>> https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version
>>
>> But it might still be easier to stick with the Build-tools versions since
>> that
>> means fewer binary packages will need to have their versions changed by
>> dh_gencontrol.
>>
>> Also, we'll need to make meta-packages for android-platform-tools and
>> android-build-tools anyway, so they can be separate source packages with
>> their
>> own versions, as needed.  These meta-packages do not contain files at all,
>> just Depends:, Recommends: etc. and other control fields.
>>
>> .hc
>>
>>
>> Hans-Christoph Steiner:
>>>
>>> Oh what a big mess they've made of this.
>>>
>>> My feeling is that we should use the Build-tools version as the source
>> package
>>> version since it is the most specific (e.g. 22.1.1) then if there are
>> updates
>>> that do not bump that version, the package version can be bumped (e.g.
>>> 22.1.1-1, 22.1.1-2, etc)
>>>
>>> Are there any utilities that use the same version number as Android OS
>>> releases (e.g. 5.1.1)?  I have not seen any.
>>>
>>> .hc
>>>
>>> 殷啟聰:
>>>> Hi Hans,
>>>>
>>>> I agree on including a copy of AndroidConfig.h in
>> android-platform-system
>>>> and document it in README.debian or README.source, that makes sense.
>>>>
>>>> On the version scheme, for example platform/dalvik, it produces
>> hprof-conv
>>>> for Platform-tools and dx for Build-tools, which will have different
>>>> versions according to SDK version scheme, then we have trouble
>> versioning
>>>> the source package android-platform-dalvik.
>>>>
>>>> Also, if you look at the difference between android-5.1.1_r7 and
>>>> android-5.1.1_r8 of platform/system/core (see <
>>>>
>> https://github.com/android/platform_system_core/compare/android-5.1.1_r7...android-5.1.1_r8
>>> ),
>>>> you can find that even 2 minor point release of Android have
>> differences on
>>>> the codes. But if you check out this 2-tag-differences on
>>>> platform/development (see <
>>>>
>> https://github.com/android/platform_development/compare/android-5.1.1_r7...android-5.1.1_r8
>>> ),
>>>> Neither the version of Platform-tools nor Build-tools has changed. This
>>>> means we can't deliver the latest tools in Android SDK if we only stick
>>>> with the SDK version scheme, also we don't know which exact tag
>> corresponds
>>>> to SDK version 22.
>>>>
>>>> In the future we will also package the emulator which will be
>> complicated
>>>> as well, so following Android versions is the best solution of following
>>>> upstream. We can still find the SDK versions in platform/development
>> under
>>>> the specific tags, and we will document that in
>> README.source/README.debian
>>>> and the package description.
>>>>
>>>> Cheers,
>>>> Kai-Chung Yan
>>>>
>>>> Hans-Christoph Steiner <hans at at.or.at> 於 2015年7月18日週六 上午12:07寫道：
>>>>
>>>>>
>>>>>
>>>>> 殷啟聰:
>>>>>> Hi,
>>>>>>
>>>>>> I found another circular dependencies! zipalign in
>> android-platform-build
>>>>>> requires libcutils.so in android-platform-system while
>>>>>> android-platform-system build-depends on AndroidConfig.h in
>>>>>> android-platform-build.
>>>>>
>>>>> Arg, why on earth did they move AndroidConfig.h?  What a pain.  I
>> think we
>>>>> should just include a copy of all the AndroidConfig.h (i.e.
>>>>> core/combo/include/arch) in android-platform-system and document that
>> in
>>>>> debian/README.Source.
>>>>>
>>>>> Then android-platform-build should make core/combo/include in a
>> separate
>>>>> package, like you said before.
>>>>>
>>>>>
>>>>>> About the version scheme, I still insist using 5.1.1.8 instead. These
>>>>>> various repo do not only contain the SDK programs, instead they are
>>>>>> actually the core components of Android. For example platform/dalvik
>>>>> which
>>>>>> contains dx, is the apparently the code base of Dalvik Virtual
>> Machine.
>>>>> As
>>>>>> for platform/system/core, it is the minimal boot environment of
>> Android
>>>>> as
>>>>>> is written. So the best solution is to follow the entire Android
>> version
>>>>>> number.
>>>>>
>>>>> It is true that these git repos include a mix of SDK Tools and
>> components
>>>>> of
>>>>> Android OS. We are not building the core components of Android itself,
>>>>> we're
>>>>> building the SDK Tools and SDK.  Upstream also builds the SDK Tools and
>>>>> Android from the same git repos, and yet upstream gives the SDK Tools
>>>>> separate
>>>>> version numbers from Android OS.
>>>>>
>>>>>
>>>>>> If you check the git history you can find that even the "SDK version"
>> has
>>>>>> remains 22 but the codes have changed a lot. Also, if you simply
>> ignore
>>>>> the
>>>>>> minor part in 22.1.3 (Build-tools version) and simply use 22 for both
>>>>>> toolsets, some day Google may have released 22.1.4 and the users still
>>>>> has
>>>>>> the old version because we have ignored the difference.
>>>>>
>>>>> We should do what upstream does.  For Build-tools, it is obvious that
>> they
>>>>> make minor (22, 22.1, etc.) and bugfix releases (22.1.1, 22.1.2, etc.).
>>>>> For
>>>>> "Android SDK Platform-tools", the only make major releases (20, 21, 22)
>>>>> though
>>>>> they have recently starting doing RC (23 rc4).  Its an insane system
>> for
>>>>> sure,
>>>>> but if we don't follow it in Debian, it will confuse users when they
>> are
>>>>> comparing to the official releases from Google.
>>>>>
>>>>> We need to track these SDK versions anyway, since
>>>>> /usr/share/android-sdk/build-tools needs to have versioned
>> subdirectories.
>>>>>
>>>>> .hc
>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Kai-Chung Yan
>>>>>>
>>>>>> 2015 年 7 月 17 日 (週五)12:15 <Hans-Christoph Steiner> 於 hans at at.or.at
>> 寫道：
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 殷啟聰:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am currently working on a new source package
>>>>>>>> "android-platform-system" based on "android-platform-system-core",
>> and
>>>>>>>> another new source package "android-platform-build". They are
>>>>>>>> currently on GitHub:
>>>>>>>>
>>>>>>>> * android-platform-system:
>>>>>>> https://github.com/seamlik/android-platform-system
>>>>>>>> * android-platform-build:
>>>>>>> https://github.com/seamlik/android-platform-build
>>>>>>>
>>>>>>> android-platform-build already exists, it should be updated:
>>>>>>>
>>>>>
>> https://anonscm.debian.org/cgit/android-tools/android-platform-build.git
>>>>>>>
>>>>>>>
>>>>>>>> Currently android-platform-build only produces one binary package:
>>>>>>>> android-globalconfig-dev, which installs AndroidConfig.h for all
>>>>>>>> architectures originating from
>>>>>>>> <
>>>>>>>
>>>>>
>> https://android.googlesource.com/platform/build/+/master/core/combo/arch>
>>>>>>>> to /usr/include/android/. The upstream of the source package is
>> simply
>>>>>>>> platform/build. These AndroidConfig.h must be included in all
>> Android
>>>>>>>> C/C++ libraries so it makes sense to become a separated package.
>>>>>>>> Packages like android-platform-system and android-platform-dalvik
>>>>>>>> should build-depends on it.
>>>>>>>>
>>>>>>>> However the mechanism of selecting which AndroidConfig.h to use is
>>>>>>>> rather unreliable for now. It uses uname -m to determine the
>>>>>>>> architecture currently, but I will use dpkg-architecture in the
>>>>>>>> future. Do you have a better idea?
>>>>>>>>
>>>>>>>> Currently android-platform-system consists of platform/system/core
>> and
>>>>>>>> platform/system/extra. The reason I chose not to make separated ones
>>>>>>>> is the circular dependencies between them if they are separated. For
>>>>>>>> example, fastboot requires f2fs_utils (in platform/system/extra),
>> and
>>>>>>>> f2fs_utils requires libsparse (in platform/system/core). There are
>>>>>>>> more components under platform/system/ but it is good to include
>> this
>>>>>>>> two for now, maybe we will need to include others like
>>>>>>>> platform/system/bluetooth in the future.
>>>>>>>>
>>>>>>>> I also restructured the Debian codes so that every modules
>>>>>>>> android-platform-system produces uses an individual .mk file, and
>> they
>>>>>>>> are not managed by Quilt.
>>>>>>>
>>>>>>> That's a nice system. :)
>>>>>>>
>>>>>>>
>>>>>>>> However there is a problem now. f2fs_utils requires another
>> libraries
>>>>>>>> from upstream f2fs-tools and e2fsprogs which are external projects
>>>>>>>> (also available at
>>>>>>>> https://android.googlesource.com/platform/external/f2fs-tools and
>>>>>>>> https://android.googlesource.com/platform/external/e2fsprogs) and
>>>>>>>> available in Debian as well (See
>>>>>>>> https://packages.debian.org/source/sid/f2fs-tools and
>>>>>>>> https://packages.debian.org/source/sid/e2fsprogs). But none of
>> their
>>>>>>>> Debian versions contains the libraries f2fs_utils requires. It
>>>>>>>> requires      .so from e2fsprogs and libf2fs_format.so from
>>>>>>>> f2fs-tools.
>>>>>>>>
>>>>>>>> The best and ideal solution is to contact with their maintainers in
>>>>>>>> Debian and make the packages produce those libraries, but it may
>> take
>>>>>>>> a long time and the required libraries may be simply unnecessary for
>>>>>>>> the upstream. The second solution is to check the upstream and the
>>>>>>>> Debian sources and see if the upstream's main libraries has already
>>>>>>>> included all APIs in libext2_uuid.so and libf2fs_format.so, but I
>>>>>>>> guess it is rather unlikly. The third solution is to make another
>> new
>>>>>>>> source package "android-platform-external" which consist of
>>>>>>>> platform/external/f2fs-tools and platform/external/e2fsprogs. This
>>>>>>>> solution is the easiest and fastest one but I don't know if it
>> breaks
>>>>>>>> any Debian policies.
>>>>>>>>
>>>>>>>> What are you opinion?
>>>>>>>
>>>>>>> If the versions of f2fs-tools and e2fsprogs in Android and Debian are
>>>>> quite
>>>>>>> close, then I think the best approach would be to make the Debian
>>>>> packages
>>>>>>> include those libraries.  If Android uses old versions or doesn't
>> update
>>>>>>> much,
>>>>>>> then I think the android-platform-external approach is better.
>>>>>>>
>>>>>>> .hc
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Kai-Chung Yan
>>>>>>>>
>>>>>>>> 2015-07-11 20:12 GMT+08:00 殷啟聰 <seamlikok at gmail.com>:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> So far we have 2 options of defining the version of
>> android-platform-*
>>>>>>>>> source packages. Suppose we are fetching sources with tag
>>>>>>>>> "android-5.1.1_r8", then the source version can be "22" or
>>>>> "1:5.1.1.8".
>>>>>>> The
>>>>>>>>> former may be  incorrect to some extent because the version of SDK
>>>>>>>>> Build-tools is 22.1.3 but the version of SDK Platform-tools is
>> 22.0.0
>>>>>>> which
>>>>>>>>> do not match. Therefore I prefer "1:5.1.1.8" which seems odd
>> though.
>>>>>>>>>
>>>>>>>>> Although androidsdk-tools is good to remain the version scheme.
>>>>>>>>>
>>>>>>>>> I am also thinking of installing symlinks to the tools in
>>>>>>>>> /usr/share/android-sdk/ which serves as the pseudo SDK installation
>>>>>>>>> location.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Kai-Chung Yan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2015 年 7 月 11 日 (週六)14:31 <Hans-Christoph Steiner> 於 hans at at.or.at
>>>>> 寫道：
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Markus Koschany:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Am 10.07.2015 um 18:42 schrieb 殷啟聰:
>>>>>>>>>>> [...]
>>>>>>>>>>>> 1. How should we define the version of the source package?
>>>>>>>>>>>> platform/dalvik produces tools belonging to both SDK Built-tools
>>>>> and
>>>>>>>>>>>> SDK Platform-tools, but this two toolsets have different version
>>>>>>>>>>>> numbers. Should we simply use "android-5.1.1_r6" as the version
>>>>>>>>>>>> number? Or "5.1.1~r6"?
>>>>>>>>>>
>>>>>>>>>> We need to find the source of the "Android SDK Tools" 24.3.3. and
>>>>>>> "Android
>>>>>>>>>> SDK
>>>>>>>>>> Build-tools" 22.0.1.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> I suppose r stands for (pre)release and the next "real" release
>> will
>>>>>>> be
>>>>>>>>>>> 5.1.2 or similar. Then it makes sense to use 5.1.1~r6 because the
>>>>>>> number
>>>>>>>>>>> is smaller than 5.1.2. This is a common pattern in Debian.
>>>>>>>>>>
>>>>>>>>>> As far as I understand it, that _r6 is actually a bugfix version,
>> so
>>>>>>> like
>>>>>>>>>> 5.1.1.6.
>>>>>>>>>>
>>>>>>>>>> The tricky part of all this is that Google uses different version
>>>>>>> schemes
>>>>>>>>>> for
>>>>>>>>>> the SDK and build tools.  For example, Android 5.1.1 is SDK
>>>>> android-22.
>>>>>>>>>> "Platform Tools" uses the SDK version e.g. 22.  Then the "Build
>>>>> Tools"
>>>>>>>>>> uses
>>>>>>>>>> that SDK version as a major version with its own minor and bugfix
>>>>>>>>>> versions.
>>>>>>>>>> But some parts of the SDK use a different major version, e.g
>> "Android
>>>>>>> SDK
>>>>>>>>>> Tools v 24.3.3".  Its a mess...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> 2. Why is the architecture of android-platform-system-core
>> packages
>>>>>>> is
>>>>>>>>>>>> "linux-any" instead of "any"?
>>>>>>>>>>
>>>>>>>>>> Upstream only supports GNU/Linux not GNU/Hurd or GNU/kFreeBSD.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> It seems the build failed at least on kfreebsd architectures.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>
>> https://buildd.debian.org/status/logs.php?pkg=android-platform-system-core&arch=kfreebsd-amd64
>>>>>>>>>>>
>>>>>>>>>>> This may be a temporary problem with the buildd, a bug in
>>>>>>>>>>> android-platform-system-core or upstream doesn't support
>>>>>>>>>>> freebsd/kfreebsd at all. I don't think the latter is true
>> because I
>>>>>>> can
>>>>>>>>>>> find several Android related ports for FreeBSD, e.g.
>>>>>>>>>>>
>>>>>>>>>>> https://www.freshports.org/devel/android-tools-adb/
>>>>>>>>>>>
>>>>>>>>>>> So the decision to build only for Linux should be reconsidered.
>>>>>>>>>>
>>>>>>>>>> There is still lots to be done, we don't need to get bogged down
>> with
>>>>>>>>>> details
>>>>>>>>>> like supporting non-Linux kernels.  We can leave the kFreeBSD
>> porting
>>>>>>> to
>>>>>>>>>> the
>>>>>>>>>> kFreeBSD people.  Let's get these packages working well for the
>> vast
>>>>>>>>>> majority
>>>>>>>>>> of people (Linux on amd64 and i386).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> 3. Are executables looks for .so files recursively under
>> /usr/lib/
>>>>> or
>>>>>>>>>>>> /lib/ ? libzipfile.so and other Android libraries are placed
>> under
>>>>>>>>>>>> /usr/lib/android/ but I do not see any specific configuration
>> for
>>>>>>>>>>>> that.
>>>>>>>>>>>
>>>>>>>>>>> You can find an dh_shlibdeps override in debian/rules. From the
>> man
>>>>>>> page
>>>>>>>>>>> of dh_shlibdeps:
>>>>>>>>>>>
>>>>>>>>>>>  -ldirectory[:directory ...]
>>>>>>>>>>> With recent versions of dpkg-shlibdeps, this option is generally
>> not
>>>>>>>>>>> needed. It tells dpkg-shlibdeps (via its -l parameter), to look
>> for
>>>>>>>>>>> private package libraries in the specified directory (or
>> directories
>>>>>>> --
>>>>>>>>>>> separate with colons). With recent versions of dpkg-shlibdeps,
>> this
>>>>> is
>>>>>>>>>>> mostly only useful for packages that build multiple flavors of
>> the
>>>>>>> same
>>>>>>>>>>> library, or other situations where the library is installed into
>> a
>>>>>>>>>>> directory not on the regular library search path.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> 4. Why the .so version is 0.21.0? For example
>> libzipfile.so.0.21.0,
>>>>>>>>>>>> why is it not libzipfile.so.21?
>>>>>>>>>>>
>>>>>>>>>>> I guess that's a question for upstream because they define the
>>>>>>>>>>> versioning. :)
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Markus
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I chose those versions.  Upstream does not build dynamic versions
>> of
>>>>>>> those
>>>>>>>>>> libraries, but only statically links that code into each
>>>>> executable.  I
>>>>>>>>>> used
>>>>>>>>>> 0.21.0 to give room to set an 'epoch' as the first version number.
>>>>>>> the 21
>>>>>>>>>> is
>>>>>>>>>> the SDK version. You can see that particular example here:
>>>>>>>>>>
>>>>>>>>>> debian/patches/libandroidzipfile_makefile_pkgconfig
>>>>>>>>>>
>>>>>>>>>> .hc
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> 殷啟聰 | Kai-Chung Yan
>>>>>>>>> 一生只向真理與妻子低頭
>>>>>>>>> Full-time student of Providence University of Taiwan
>>>>>>>>> LinkedIn: <https://linkedin.com/in/seamlik>
>>>>>>>>> Blog: <seamlik.logdown.com>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>