[Debian-med-packaging] Bug#954270: [RFS] kmc: arm64 autopkgtest time out

Thu Dec 10 12:50:59 GMT 2020

Hi

On 10-12-2020 13:10, Andreas Tille wrote:
> On Thu, Dec 10, 2020 at 11:58:23AM +0100, Paul Gevers wrote:
>>> In the kmc case I'm seriously wondering whether we should restrict
>>> the architectures to those that are relevant in practice.  It seems
>>> to be in line with upstream and I'm tempted to follow Étienne's
>>> suggestion to upgrade to kmc 3.x which does not even build on armhf
>>> any more and remove the package for the non-building architectures.
>>>
>>> Étienne, would you mind pushing your patches to Git to proceed with
>>> this plan?
>>
>> Shall I run the current package on our big amd64 host to check if it
>> fails there and that part of problem is solved? I mean, I understand
>> that there was a real problem with hosts that have lots of CPU's and I
>> assume you support amd64.
> 
> My guess is that kmc was developed under and designed for amd64 and it
> runs there.  That's explicitly the case for the latest upstream version
> which will not support any arm* out of the box (if I understand Étienne
> correctly).  I do not see any need to stress test our hardware with
> some software that will be outdated soonish.
> 
> Kind regards
> 
>        Andreas.
> 
> 

I couldn't resist. I started it up and it hangs.

Paul

root at ci-worker13:~# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       46 bits physical, 48 bits virtual
CPU(s):              48
On-line CPU(s) list: 0-47
Thread(s) per core:  2
Core(s) per socket:  12
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               79
Model name:          Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Stepping:            1
CPU MHz:             2200.074
CPU max MHz:         2900.0000
CPU min MHz:         1200.0000
BogoMIPS:            4400.13
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            30720K
NUMA node0 CPU(s):   0-11,24-35
NUMA node1 CPU(s):   12-23,36-47
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor
ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1
sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand
lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single
pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept
vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm
cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc
cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts flush_l1d

root at ci-worker13:~# /usr/bin/autopkgtest --no-built-binaries
'--setup-commands=echo '"'"'kmc testing/amd64'"'"' > /var/tmp/debci.pkg
2>&1 || true' --user debci --apt-upgrade '--add-apt-source=deb
http://incoming.debian.org/debian-buildd buildd-unstable main contrib
non-free' --add-apt-release=unstable --pin-packages=unstable=src:kmc kmc
-- lxc --sudo --name elbrus autopkgtest-testing-amd64

[...]

autopkgtest [12:45:12]: test build-lib: [-----------------------
g++ kmc_dump/*cpp -std=c++11 -lkmc -o $WORKDIR/my_kmcdump
[ -x $WORKDIR/my_kmcdump ]
echo "build: OK"

cd $WORKDIR
echo 'Running kmc (single threaded)'
kmc -ci1 -m1 -k28 -t1 $ORIGDIR/debian/tests/sample_6.fastq.gz 1 .
build: OK
Running kmc (single threaded)
*
*
********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
1st stage: 0.170529s
2nd stage: 0.041426s
Total    : 0.211955s
Tmp size : 0MB

Stats:
   No. of k-mers below min. threshold :            0
   No. of k-mers above max. threshold :            0
   No. of unique k-mers               :         6435
   No. of unique counted k-mers       :         6435
   Total no. of k-mers                :         6714
   Total no. of reads                 :          750
   Total no. of super-k-mers          :         1234
ls -Al
total 1200
-rw-r--r-- 1 debci debci 1114204 Dec 10 12:45 1.kmc_pre
-rw-r--r-- 1 debci debci   45053 Dec 10 12:45 1.kmc_suf
-rwxr-xr-x 1 debci debci   63024 Dec 10 12:45 my_kmcdump
[ -s 1.kmc_suf ]
[ -s 1.kmc_pre ]
echo "kmc (single threaded): OK"
kmc (single threaded): OK

./my_kmcdump 1 out
[ -s out ]
echo "run: OK"
run: OK

# FIXME: uncomment the below test once kmc is updated to version 3.1.1
or more.
## Multi-threaded runs have been known to be faulty (#954270).
#echo 'Running kmc (32 threads)'
#rm -f 1.kmc_suf 1.kmc_pre
#kmc -ci1 -m1 -k28 -t32 $ORIGDIR/debian/tests/sample_6.fastq.gz 1 .
#ls -Al
#[ -s 1.kmc_suf ]
#[ -s 1.kmc_pre ]
#echo "kmc (32 threads): OK"

# In the end, default options should, ideally, work on any configuration.
echo 'Running kmc (default thread count)'
Running kmc (default thread count)
rm -f 1.kmc_suf 1.kmc_pre
kmc -ci1 -m1 -k28 $ORIGDIR/debian/tests/sample_6.fastq.gz 1 .
*

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/debian-med-packaging/attachments/20201210/53bdbff2/attachment.sig>