[Pkg-dpdk-devel] Bug#917984: Bug#917984: Bug#917984: Bug#917984: Can not link ODP with newer DPDK

[Luca Boccassi]

On Thu, 2019-01-03 at 02:29 +0300, Dmitry Eremin-Solenikov wrote:
> Hello,
> 
> чт, 3 янв. 2019 г. в 02:09, Luca Boccassi <bluca at debian.org>:
> > On Wed, 2019-01-02 at 14:55 +0300, Dmitry Eremin-Solenikov wrote:
> > > ср, 2 янв. 2019 г. в 14:49, Luca Boccassi <bluca at debian.org>:
> > > > On Wed, 2019-01-02 at 01:43 +0300, Dmitry Eremin-Solenikov
> > > > wrote:
> > > > Strange that libtool is messing things up, I've used the same
> > > > pkgconfig
> > > > file in a few different projects that use autoconf/automake and
> > > > I
> > > > haven't seen this issue.
> > > 
> > > libtool rearranges/squashes linking flags in an attempt to find
> > > 'better'
> > > linking flags. Unfortunately this fail for DPDK. We have worked
> > > around
> > > this by squashing all PMDs into a single gcc argument:
> > > -Wl,--whole-archive,-lrte_pmd_af_packet,-lrte_pmd_ark,........,-
> > > lrte_pmd_vmxnet3_uio,--no-whole-archive
> > > -ldpdk
> > > 
> > > Thus libtool won't move PMDs from --whole-archive/--no-whole-
> > > archive
> > > brackets.
> > > 
> > > > I had a look on github, and it does not seem that odp is
> > > > currently
> > > > using pkg-config, but rather doing some manual check - is there
> > > > a
> > > > branch in a fork or a patch you could point me to so that I can
> > > > try
> > > > to
> > > > reproduce?
> > > 
> > > No, I have not pushed my code to github yet. The easies way to
> > > reproduce
> > > is to statically link a sample program with libtool and check
> > > that
> > > generated
> > > ELF contains all PMDs.
> > 
> > That looks like a very very old libtool bug:
> > 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=347650
> > 
> > Have you tried patching config/ltmain.sh as it's suggested on that
> > bug?
> 
> I can try doing that as a test, but I wouldn't like to have patched
> ltmain.sh
> in the source tree.

Another workaround has been suggested and I've verified that it works:
parse the output of pkg-config --libs --static libdpdk and change the
string "-Wl,--whole-archive -lrte_pmdfoo -lrte_pmdbar ... -Wl,--no-
whole-archive" intoto "-Wl,--whole-archive,-lrte_pmdfoo,-
lrte_pmdbar,...,--no-whole-archive"

Basically, trick libtool into thinking that it's a single linker flag:

libtool --mode=link --tag=CC gcc test.o -o a.out -lrte_telemetry -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_vhost -lrte_security -lrte_sched -lrte_reorder -lrte_rawdev -lrte_pdump -lrte_power -lrte_meter -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_pci -lrte_ethdev -lrte_net -lrte_mbuf -lrte_mempool -lrte_ring -lrte_eal -lrte_kvargs -lrte_cmdline -Wl,--whole-archive,-lrte_common_cpt,-lrte_common_dpaax,-lrte_common_octeontx,-lrte_bus_dpaa,-lrte_bus_fslmc,-lrte_bus_ifpga,-lrte_bus_pci,-lrte_bus_vdev,-lrte_bus_vmbus,-lrte_mempool_bucket,-lrte_mempool_dpaa,-lrte_mempool_dpaa2,-lrte_mempool_octeontx,-lrte_mempool_ring,-lrte_mempool_stack,-lrte_pmd_af_packet,-lrte_pmd_ark,-lrte_pmd_atlantic,-lrte_pmd_avf,-lrte_pmd_avp,-lrte_pmd_axgbe,-lrte_pmd_bond,-lrte_pmd_bnx2x,-lrte_pmd_bnxt,-lrte_pmd_cxgbe,-lrte_pmd_dpaa,-lrte_pmd_dpaa2,-lrte_pmd_e1000,-lrte_pmd_ena,-lrte_pmd_enetc,-lrte_pmd_enic,-lrte_pmd_failsafe,-lrte_pmd_fm10k,-lrte_pmd_i40e,-lrte_pmd_ifc,-lrte_pmd_ixgbe,-lrte_pmd_kni,-lrte_pmd_liquidio,-lrte_pmd_mlx4,-lrte_pmd_mlx5,-lrte_pmd_netvsc,-lrte_pmd_nfp,-lrte_pmd_null,-lrte_pmd_octeontx,-lrte_pmd_pcap,-lrte_pmd_qede,-lrte_pmd_ring,-lrte_pmd_sfc,-lrte_pmd_softnic,-lrte_pmd_tap,-lrte_pmd_thunderx,-lrte_pmd_vdev_netvsc,-lrte_pmd_vhost,-lrte_pmd_virtio,-lrte_pmd_vmxnet3,-lrte_pmd_aesni_gcm,-lrte_pmd_aesni_mb,-lrte_pmd_caam_jr,-lrte_pmd_ccp,-lrte_pmd_dpaa_sec,-lrte_pmd_dpaa2_sec,-lrte_pmd_null_crypto,-lrte_pmd_octeontx_crypto,-lrte_pmd_openssl,-lrte_pmd_crypto_scheduler,-lrte_pmd_virtio_crypto,-lrte_pmd_octeontx_compress,-lrte_pmd_qat,-lrte_pmd_zlib,-lrte_pmd_dpaa_event,-lrte_pmd_dpaa2_event,-lrte_pmd_octeontx_event,-lrte_pmd_opdl_event,-lrte_pmd_skeleton_event,-lrte_pmd_sw_event,-lrte_pmd_dsw_event,-lrte_pmd_bbdev_null,-lrte_pmd_skeleton_rawdev,-lrte_pmd_dpaa2_cmdif,-lrte_pmd_dpaa2_qdma,-lrte_pmd_ifpga_rawdev,--no-whole-archive -lrte_telemetry -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_vhost -lrte_security -lrte_sched -lrte_reorder -lrte_rawdev -lrte_pdump -lrte_power -lrte_meter -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_pci -lrte_ethdev -lrte_net -lrte_mbuf -lrte_mempool -lrte_ring -lrte_eal -lrte_kvargs -lrte_cmdline -Wl,-Bdynamic -Wl,--no-as-needed -pthread -lm -ldl -lnuma -lz -lIPSec_MB -lpcap -lbsd -lmnl -lmlx4 -lpthread -lmlx5 -lpthread -libverbs -lbnxt_re-rdmav21 -lcxgb3-rdmav21 -lcxgb4-rdmav21 -lhns-rdmav21 -li40iw-rdmav21 -lmlx4 -lmlx5 -lmthca-rdmav21 -lnes-rdmav21 -locrdma-rdmav21 -lqedr-rdmav21 -lvmw_pvrdma-rdmav21 -lhfi1verbs-rdmav21 -lipathverbs-rdmav21 -lrxe-rdmav21 -libverbs -lpthread -lnl-route-3 -lnl-3 -lcrypto -ldl -pthread -ljansson -lelf -lz -all-static
libtool: link: gcc test.o -o a.out -Wl,--whole-archive -Wl,-lrte_common_cpt -Wl,-lrte_common_dpaax -Wl,-lrte_common_octeontx -Wl,-lrte_bus_dpaa -Wl,-lrte_bus_fslmc -Wl,-lrte_bus_ifpga -Wl,-lrte_bus_pci -Wl,-lrte_bus_vdev -Wl,-lrte_bus_vmbus -Wl,-lrte_mempool_bucket -Wl,-lrte_mempool_dpaa -Wl,-lrte_mempool_dpaa2 -Wl,-lrte_mempool_octeontx -Wl,-lrte_mempool_ring -Wl,-lrte_mempool_stack -Wl,-lrte_pmd_af_packet -Wl,-lrte_pmd_ark -Wl,-lrte_pmd_atlantic -Wl,-lrte_pmd_avf -Wl,-lrte_pmd_avp -Wl,-lrte_pmd_axgbe -Wl,-lrte_pmd_bond -Wl,-lrte_pmd_bnx2x -Wl,-lrte_pmd_bnxt -Wl,-lrte_pmd_cxgbe -Wl,-lrte_pmd_dpaa -Wl,-lrte_pmd_dpaa2 -Wl,-lrte_pmd_e1000 -Wl,-lrte_pmd_ena -Wl,-lrte_pmd_enetc -Wl,-lrte_pmd_enic -Wl,-lrte_pmd_failsafe -Wl,-lrte_pmd_fm10k -Wl,-lrte_pmd_i40e -Wl,-lrte_pmd_ifc -Wl,-lrte_pmd_ixgbe -Wl,-lrte_pmd_kni -Wl,-lrte_pmd_liquidio -Wl,-lrte_pmd_mlx4 -Wl,-lrte_pmd_mlx5 -Wl,-lrte_pmd_netvsc -Wl,-lrte_pmd_nfp -Wl,-lrte_pmd_null -Wl,-lrte_pmd_octeontx -Wl,-lrte_pmd_pcap -Wl,-lrte_pmd_qede -Wl,-lrte_pmd_ring -Wl,-lrte_pmd_sfc -Wl,-lrte_pmd_softnic -Wl,-lrte_pmd_tap -Wl,-lrte_pmd_thunderx -Wl,-lrte_pmd_vdev_netvsc -Wl,-lrte_pmd_vhost -Wl,-lrte_pmd_virtio -Wl,-lrte_pmd_vmxnet3 -Wl,-lrte_pmd_aesni_gcm -Wl,-lrte_pmd_aesni_mb -Wl,-lrte_pmd_caam_jr -Wl,-lrte_pmd_ccp -Wl,-lrte_pmd_dpaa_sec -Wl,-lrte_pmd_dpaa2_sec -Wl,-lrte_pmd_null_crypto -Wl,-lrte_pmd_octeontx_crypto -Wl,-lrte_pmd_openssl -Wl,-lrte_pmd_crypto_scheduler -Wl,-lrte_pmd_virtio_crypto -Wl,-lrte_pmd_octeontx_compress -Wl,-lrte_pmd_qat -Wl,-lrte_pmd_zlib -Wl,-lrte_pmd_dpaa_event -Wl,-lrte_pmd_dpaa2_event -Wl,-lrte_pmd_octeontx_event -Wl,-lrte_pmd_opdl_event -Wl,-lrte_pmd_skeleton_event -Wl,-lrte_pmd_sw_event -Wl,-lrte_pmd_dsw_event -Wl,-lrte_pmd_bbdev_null -Wl,-lrte_pmd_skeleton_rawdev -Wl,-lrte_pmd_dpaa2_cmdif -Wl,-lrte_pmd_dpaa2_qdma -Wl,-lrte_pmd_ifpga_rawdev -Wl,--no-whole-archive -Wl,-Bdynamic -Wl,--no-as-needed -pthread -pthread -static  -lrte_telemetry -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_vhost -lrte_security -lrte_sched -lrte_reorder -lrte_rawdev -lrte_pdump -lrte_power -lrte_meter -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_pci -lrte_ethdev -lrte_net -lrte_mbuf -lrte_mempool -lrte_ring -lrte_eal -lrte_kvargs -lrte_cmdline -lm -lnuma -lIPSec_MB -lpcap -lbsd -lmnl -lbnxt_re-rdmav21 -lcxgb3-rdmav21 -lcxgb4-rdmav21 -lhns-rdmav21 -li40iw-rdmav21 -lmlx4 -lmlx5 -lmthca-rdmav21 -lnes-rdmav21 -locrdma-rdmav21 -lqedr-rdmav21 -lvmw_pvrdma-rdmav21 -lhfi1verbs-rdmav21 -lipathverbs-rdmav21 -lrxe-rdmav21 -libverbs -lpthread -lnl-route-3 -lnl-3 -lcrypto -ldl -ljansson -lelf -lz -pthread

I'll see if I can get meson to generate that directly, but I don't
think it's possible from what I can see, so I'd suggest to employ one
of these workarounds. Or drop libtool :-)

> > Something like:
> 
> [patch skipped]
> 
> > Note that the current version of Meson does not do a good job of
> > generating the pkg-config file, but it's fixed in the version in
> > development. I also found a couple of bugs in dpdk. So the
> > following
> > content for libdpdk.pc is more correct:
> 
> [libdpdk.pc skipped]
> 
> Do you plan to upload fixed dpdk packages?

Yes, I'll push the changes upstream and backport them sometimes within
the next couple of days.

> > With that I can manually do a static link (without using libtool).
> 
> Good!
> 
> BTW: Is there any chance to get libdpdk.a back? We can then work
> on fixing linking with libdpdk.pc as the time permits. Note:
> according
> to README.md the 'official' DPDK build is one done using GNU Make
> and this build has libdpdk.a instead of libdpdk.pc.

The linker script and single archive were a bit of a hack and are not
planned to be supported under Meson as far as I'm aware, so I'm afraid
not.

-- 
Kind regards,
Luca Boccassi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://alioth-lists.debian.net/pipermail/pkg-dpdk-devel/attachments/20190103/d08f7a45/attachment.sig>