通过EUR环境迁移打包开源分布式存储软件Lizardfs到openEuler系统的问题解决记录

Lizardfs项目官网及github:
https://lizardfs.com

在EUR中的项目地址:
https://eur.openeuler.openatom.cn/coprs/kerneltravel/lizardfs

简介:Lizardfs 是由波兰开发者发起的一种分布式文件存储领域的开源软件,具有分布式软件的主流功能,兼顾性能与稳定性,相比ceph具有极高的易用性。期待社区有更多小伙伴了解Lizardfs,一起研究和改进Lizardfs及成为分布式存储领域的优秀的开源方案之一。

EUR 是openatom基于 GitHub - fedora-copr/copr: RPM build system - upstream for https://copr.fedorainfracloud.org/ 建立的RPM打包系统(RPM build system),类似的系统在Fedora社区也有(https://copr.fedorainfracloud.org )。

项目目的是将 开源分布式存储软件 lizardfs 在openEuler 打包验证。正好近期EUR功能上线,所以在EUR中做这个打包过程。

由于第一次使用EUR,之前遇到了几个基本问题(如网络要enable 才能clone到github.com 的仓库代码、以及 srpm 的 spec 不存在等问题,经过社区大佬的指点已正常 )。

本帖将把迁移打包开源分布式存储软件Lizardfs到openEuler系统的过程中遇到的各类问题和解决做记录,以帮助后面有类似目标的社区同学在迁移过程中更顺畅。也希望这个优秀的开源分布式存储软件Lizardfs最终在openEuler社区能被更多人关注到它的价值。

1 个赞

支持支持

今天做的一些改进:

为了git clone 时网络更稳定一些,使用了 GitHub - lizardfs/lizardfs: LizardFS is an Open Source Distributed File System licensed under GPLv3. 镜像 到gitee.com 的 lizardfs 仓库:lizardfs: https://github.com/lizardfs/lizardfs 的国内git镜像,用于openEuler的适配打包 作为代码来源。

由于lizardfs 仓库里有提供了rpm 打包的shell脚本,
lizardfs 官方的jenkins CI里有一个 使用 create-rpm-package.sh 脚本构建rpm包的 过程记录:
lizardfs.BuildPackages » centos-8 #2942 Console [Jenkins]

文件内容为 create-rpm-package.sh · kjpioo/lizardfs - Gitee.com
所以现在在EUR中将packages 的 Select the source type 改为 custom 方式。

目前该脚本的内容为:

#!/bin/sh -x
set -eux

#curl https://github.com/lizardfs/lizardfs/rpm/lizardfs.spec -O

#os_release=“$(cat /etc/redhat-release | tr -dc ‘0-9’ | cut -b 1-2)”

# case “$os_release” in

# 7*) distro=el7 ;;

# 8*) distro=el8 ;;

# 34*) distro=fc34 ;;

# 35*) distro=fc35 ;;

# *) echo “This Linux distribution is not supported” && exit 1 # # ;;

#esac

distro=el8
echo “Building packages for ‘$distro’”

#lizardfs git repo URL
git_repo_url=“lizardfs: https://github.com/lizardfs/lizardfs 的国内git镜像,用于openEuler的适配打包

Directories used by this script

output_dir=$(pwd)
source_dir=/tmp/lizardfs_source_dir
working_dir=/tmp/lizardfs_rpm_working_directory

Create an empty working directory for rpmbuild

rm -rf “$working_dir”
mkdir -p “$working_dir”/{BUILD,SOURCES,SPECS,RPMS,SRPMS}
mkdir -p “$source_dir”

Create a source tarball (the same as those available on Github) using git-archive to make

sure there are no additional files included in the source package

cd “$source_dir”
git clone $git_repo_url .

version=$(git show HEAD:rpm/lizardfs.spec | awk ‘/^Version:/ {print $2}’)
release=“lizardfs-$version”
git archive --prefix=“$release”/ --format=tar HEAD | gzip > “$working_dir/SOURCES/$release.tar.gz”
git show HEAD:rpm/lizardfs.spec | sed -e “s/@DISTRO@/$distro/” > “$working_dir/SPECS/lizardfs.spec”

Install dependent package manully

rpm -ivh https://gitee.com/kjpioo2006/fmt/raw/master/root/rpmbuild/RPMS/x86_64/fmt-8.0.1-2.x86_64.rpm
rpm -ivh https://gitee.com/kjpioo2006/fmt/raw/master/root/rpmbuild/RPMS/x86_64/fmt-devel-8.0.1-2.x86_64.rpm

Build packages

cd “$working_dir/SPECS”
rpmbuild --define “_topdir $working_dir” -ba lizardfs.spec

Copy all the created files and clean up

cp “$working_dir”/{RPMS/x86_64,SRPMS}/* “$output_dir”
#rm -rf “$working_dir”

今天在EUR中按以上脚本编译时,由于openEuler 22.03 中没有可供直接yum方式安装的fmt 包,所以我从源码仓库 制作了 fmt 包 fmt: Small, safe and fast formatting library for C++. ,但目前仍报错:

https://eur.openeuler.openatom.cn/results/kerneltravel/lizardfs/srpm-builds/00002791/builder-live.log.gz

……
– Performing Test LIZARDFS_HAVE_CPU_CHECK - Success
– Looking for FALLOC_FL_PUNCH_HOLE
– Looking for FALLOC_FL_PUNCH_HOLE - found
– Performing Test LIZARDFS_HAVE_STD_FUTURE
– Performing Test LIZARDFS_HAVE_STD_FUTURE - Success
CMake Error at cmake/Libraries.cmake:9 (find_package):
By not providing “Findfmt.cmake” in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by “fmt”, but
CMake did not find one.

Could not find a package configuration file provided by “fmt” with any of
the following names:

fmtConfig.cmake
fmt-config.cmake

Add the installation prefix of “fmt” to CMAKE_PREFIX_PATH or set “fmt_DIR”
to a directory containing one of the above files. If “fmt” provides a
separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
CMakeLists.txt:207 (include)

– Configuring incomplete, errors occurred!
See also “/tmp/lizardfs_rpm_working_directory/BUILD/lizardfs-3.13.0/build-pack/CMakeFiles/CMakeOutput.log”.
See also “/tmp/lizardfs_rpm_working_directory/BUILD/lizardfs-3.13.0/build-pack/CMakeFiles/CMakeError.log”.
error: Bad exit status from /var/tmp/rpm-tmp.7VgZxn (%build)

RPM build errors:
Macro expanded in comment on line 398: %{_libdir}/ganesha/libfsallizardfs.so

Bad exit status from /var/tmp/rpm-tmp.7VgZxn (%build)

Finish: chroot [‘su - mockbuild -c 'set -xe ; cd /workdir ; COPR_OWNER=kerneltravel COPR_PROJECT=lizardfs COPR_PACKAGE='“'”''“'”' COPR_RESULTDIR=/workdir /script'’]
Traceback (most recent call last):
File “/usr/bin/copr-sources-custom”, line 115, in
run_cmd(mock + [‘–chroot’, ‘su - {0} -c {1}’.format(user, cmd)])
File “/usr/bin/copr-sources-custom”, line 64, in run_cmd
return subprocess.check_call(command, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib64/python3.11/subprocess.py”, line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command ‘[‘mock’, ‘-r’, ‘/var/lib/copr-rpmbuild/results/mock-config.cfg’, ‘–chroot’, ‘su - mockbuild -c 'set -xe ; cd /workdir ; COPR_OWNER=kerneltravel COPR_PROJECT=lizardfs COPR_PACKAGE='“'”''“'”' COPR_RESULTDIR=/workdir /script'’]’ returned non-zero exit status 1.
Copr build error: Build failed
Shared connection to 10.0.0.62 closed.

请问老师,在EUR 里目前这种情况,有比较好的安装 “fmt” 和 “fmt-devel” 的方法吗?

现在提示找不到
fmtConfig.cmake
fmt-config.cmake

openEuler 22.03里是有包含fmt-devel,但是 lizardfs.spec里没有 BuildRequires: cmake(fmt),可以加上再试试,另外,自己打包的fmt-devel是没有包含fmtConfig.cmake的,因此会报错

按老师的提示改进了,现在这个fmt的报错没了。

继续处理下一个依赖包 spdlog 和spdlog-devel (在openEuler仓库中没有rpm)

  • STATUS=0

  • ‘[’ 0 -ne 0 ‘]’

  • cd lizardfs-3.13.0

  • /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .

  • RPM_EC=0
    ++ jobs -p

  • exit 0
    Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.IcgQFs

  • umask 022

  • cd /tmp/lizardfs_rpm_working_directory/BUILD

  • cd lizardfs-3.13.0

  • ./configure --with-doc
    CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
    Compatibility with CMake < 2.8.12 will be removed from a future version of
    CMake.

    Update the VERSION argument value or use a … suffix to tell
    CMake that the project does not need compatibility with older versions.

– ENABLE_CCACHE: ON
– CCACHE_FOUND: CCACHE_FOUND-NOTFOUND
– The C compiler identification is GNU 10.3.1
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: /usr/bin/cc - skipped
– Detecting C compile features
– Detecting C compile features - done
CMake Warning (dev) at CMakeLists.txt:93 (set):
implicitly converting ‘INT’ to ‘STRING’ type.
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) at CMakeLists.txt:94 (set):
implicitly converting ‘INT’ to ‘STRING’ type.
This warning is for project developers. Use -Wno-dev to suppress it.

– CMAKE_BUILD_TYPE: Release
– CMAKE_INSTALL_PREFIX: /
– DEFAULT_USER: lizardfs
– DEFAULT_GROUP: lizardfs
– LIZARDFS_BLOCKS_IN_CHUNK: 1024
– LIZARDFS_BLOCK_SIZE: 65536
– ENABLE_WERROR: OFF
– ENABLE_LIGHTMFS: OFF
– ENABLE_UTILS: OFF
– ENABLE_TESTS: NO
– ENABLE_DOCS: YES
– ENABLE_EXIT_ON_USR1: OFF
– THROW_INSTEAD_OF_ABORT: OFF
– ENABLE_DEBUG_LOG: OFF
– ENABLE_VERBOSE_ASCIIDOC: OFF
– ENABLE_TCMALLOC: OFF
– ENABLE_JEMALLOC: OFF
– ENABLE_POLONAISE: OFF
– ENABLE_CLIENT_LIB: YES
– ENABLE_OFFICIAL_BUILD: NO
– SET_RC_BUILD_NUMBER: NO
– ENABLE_URAFT: YES
– ENABLE_TRACES: OFF
– ENABLE_CRC: ON
– ENABLE_REQUEST_LOG: OFF
– USE_LEGACY_READ_MESSAGES: OFF
– ENABLE_NFS_GANESHA: NO
fatal: not a git repository (or any of the parent directories): .git
– The CXX compiler identification is GNU 10.3.1
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/c++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Looking for include file arpa/inet.h
– Looking for include file arpa/inet.h - found
– Looking for include file fcntl.h
– Looking for include file fcntl.h - found
– Looking for include file inttypes.h
– Looking for include file inttypes.h - found
– Looking for include file limits.h
– Looking for include file limits.h - found
– Looking for include file netdb.h
– Looking for include file netdb.h - found
– Looking for include file netinet/in.h
– Looking for include file netinet/in.h - found
– Looking for include file stddef.h
– Looking for include file stddef.h - found
– Looking for include file stdlib.h
– Looking for include file stdlib.h - found
– Looking for include file string.h
– Looking for include file string.h - found
– Looking for include file sys/mman.h
– Looking for include file sys/mman.h - found
– Looking for include file sys/resource.h
– Looking for include file sys/resource.h - found
– Looking for include file sys/rusage.h
– Looking for include file sys/rusage.h - not found
– Looking for include file sys/socket.h
– Looking for include file sys/socket.h - found
– Looking for include file sys/statvfs.h
– Looking for include file sys/statvfs.h - found
– Looking for include file sys/time.h
– Looking for include file sys/time.h - found
– Looking for include file syslog.h
– Looking for include file syslog.h - found
– Looking for include file unistd.h
– Looking for include file unistd.h - found
– Looking for include file stdbool.h
– Looking for include file stdbool.h - found
– Looking for include file isa-l/erasure_code.h
– Looking for include file isa-l/erasure_code.h - not found
Some include files were not found: sys/rusage.h;isa-l/erasure_code.h
– Looking for sys/types.h
– Looking for sys/types.h - found
– Looking for stdint.h
– Looking for stdint.h - found
– Looking for stddef.h
– Looking for stddef.h - found
– Check size of int8_t
– Check size of int8_t - done
– Check size of uint8_t
– Check size of uint8_t - done
– Check size of int16_t
– Check size of int16_t - done
– Check size of uint16_t
– Check size of uint16_t - done
– Check size of int32_t
– Check size of int32_t - done
– Check size of uint32_t
– Check size of uint32_t - done
– Check size of int64_t
– Check size of int64_t - done
– Check size of uint64_t
– Check size of uint64_t - done
– Check size of uid_t
– Check size of uid_t - done
– Check size of mode_t
– Check size of mode_t - done
– Check size of off_t
– Check size of off_t - done
– Check size of pid_t
– Check size of pid_t - done
– Check size of size_t
– Check size of size_t - done
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_BLOCKS
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_BLOCKS - Success
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_RDEV
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_RDEV - Success
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_BIRTHTIME
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_BIRTHTIME - Failed
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_BLKSIZE
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_BLKSIZE - Success
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_FLAGS
– Performing Test LIZARDFS_HAVE_STRUCT_STAT_ST_FLAGS - Failed
– Performing Test LIZARDFS_HAVE_STRUCT_TM_TM_GMTOFF
– Performing Test LIZARDFS_HAVE_STRUCT_TM_TM_GMTOFF - Failed
– Performing Test LIZARDFS_HAVE_STRUCT_RUSAGE_RU_MAXRSS
– Performing Test LIZARDFS_HAVE_STRUCT_RUSAGE_RU_MAXRSS - Success
– Looking for fork
– Looking for fork - found
– Looking for vfork
– Looking for vfork - found
– Performing Test LIZARDFS_HAVE_STD_TO_STRING
– Performing Test LIZARDFS_HAVE_STD_TO_STRING - Success
– Performing Test LIZARDFS_HAVE_STD_STOULL
– Performing Test LIZARDFS_HAVE_STD_STOULL - Success
– Looking for atexit
– Looking for atexit - found
– Looking for ftruncate
– Looking for ftruncate - found
– Looking for gettimeofday
– Looking for gettimeofday - found
– Looking for memmove
– Looking for memmove - found
– Looking for memset
– Looking for memset - found
– Looking for mkdir
– Looking for mkdir - found
– Looking for strchr
– Looking for strchr - found
– Looking for strdup
– Looking for strdup - found
– Looking for strtol
– Looking for strtol - found
– Looking for strtoul
– Looking for strtoul - found
– Looking for ftello
– Looking for ftello - found
– Looking for fseeko
– Looking for fseeko - found
– Looking for getpass
– Looking for getpass - found
– Looking for poll
– Looking for poll - found
– Looking for realpath
– Looking for realpath - found
– Looking for strerror
– Looking for strerror - found
– Looking for perror
– Looking for perror - found
– Looking for pread
– Looking for pread - found
– Looking for pwrite
– Looking for pwrite - found
– Looking for readv
– Looking for readv - found
– Looking for writev
– Looking for writev - found
– Looking for getrusage
– Looking for getrusage - found
– Looking for setitimer
– Looking for setitimer - found
– Looking for posix_fadvise
– Looking for posix_fadvise - found
– Looking for fallocate
– Looking for fallocate - found
– Looking for clock_gettime in rt
– Looking for clock_gettime in rt - found
– Looking for dup2
– Looking for dup2 - found
– Looking for mlockall
– Looking for mlockall - found
– Looking for getcwd
– Looking for getcwd - found
– Performing Test LIZARDFS_HAVE_STD_CHRONO_STEADY_CLOCK
– Performing Test LIZARDFS_HAVE_STD_CHRONO_STEADY_CLOCK - Success
– Performing Test LIZARDFS_HAVE_STD_ALLOCATOR_TRAITS
– Performing Test LIZARDFS_HAVE_STD_ALLOCATOR_TRAITS - Success
– Performing Test LIZARDFS_HAVE_THREAD_LOCAL
– Performing Test LIZARDFS_HAVE_THREAD_LOCAL - Success
– Performing Test CXX_HAS_MCRC32
– Performing Test CXX_HAS_MCRC32 - Success
– Performing Test LIZARDFS_HAVE_CPU_CHECK
– Performing Test LIZARDFS_HAVE_CPU_CHECK - Success
– Looking for FALLOC_FL_PUNCH_HOLE
– Looking for FALLOC_FL_PUNCH_HOLE - found
– Performing Test LIZARDFS_HAVE_STD_FUTURE
– Performing Test LIZARDFS_HAVE_STD_FUTURE - Success
CMake Error at cmake/Libraries.cmake:10 (find_package):
By not providing “Findspdlog.cmake” in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by “spdlog”, but
CMake did not find one.

Could not find a package configuration file provided by “spdlog” with any
of the following names:

spdlogConfig.cmake
spdlog-config.cmake

Add the installation prefix of “spdlog” to CMAKE_PREFIX_PATH or set
“spdlog_DIR” to a directory containing one of the above files. If “spdlog”
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
CMakeLists.txt:207 (include)

– Configuring incomplete, errors occurred!
See also “/tmp/lizardfs_rpm_working_directory/BUILD/lizardfs-3.13.0/build-pack/CMakeFiles/CMakeOutput.log”.
See also “/tmp/lizardfs_rpm_working_directory/BUILD/lizardfs-3.13.0/build-pack/CMakeFiles/CMakeError.log”.
error: Bad exit status from /var/tmp/rpm-tmp.IcgQFs (%build)

RPM build errors:
Macro expanded in comment on line 400: %{_libdir}/ganesha/libfsallizardfs.so

Bad exit status from /var/tmp/rpm-tmp.IcgQFs (%build)

Finish: chroot [‘su - mockbuild -c 'set -xe ; cd /workdir ; COPR_OWNER=kerneltravel COPR_PROJECT=lizardfs COPR_PACKAGE='“'”''“'”' COPR_RESULTDIR=/workdir /script'’]
Traceback (most recent call last):
File “/usr/bin/copr-sources-custom”, line 115, in
run_cmd(mock + [‘–chroot’, ‘su - {0} -c {1}’.format(user, cmd)])
File “/usr/bin/copr-sources-custom”, line 64, in run_cmd
return subprocess.check_call(command, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib64/python3.11/subprocess.py”, line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command ‘[‘mock’, ‘-r’, ‘/var/lib/copr-rpmbuild/results/mock-config.cfg’, ‘–chroot’, ‘su - mockbuild -c 'set -xe ; cd /workdir ; COPR_OWNER=kerneltravel COPR_PROJECT=lizardfs COPR_PACKAGE='“'”''“'”' COPR_RESULTDIR=/workdir /script'’]’ returned non-zero exit status 1.
Copr build error: Build failed

Finish: run
INFO:main:running command: mock -r /var/lib/copr-rpmbuild/results/mock-config.cfg --install spdlog spdlog-devel fmt-devel fmt rpm git procps-ng fuse-devel cmake asciidoc systemd rpm-build zlib-devel gcc gcc-c++ boost-devel boost
INFO: mock.py version 3.5 starting (python version = 3.11.1, NVR = mock-3.5-1.fc37)…
Start(bootstrap): init plugins
INFO: selinux disabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux disabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: enabled HW Info plugin
Mock Version: 3.5
INFO: Mock Version: 3.5
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
Mock Version: 3.5
INFO: Mock Version: 3.5
Finish: chroot init
INFO: installing package(s): spdlog spdlog-devel fmt-devel fmt rpm git procps-ng fuse-devel cmake asciidoc systemd rpm-build zlib-devel gcc gcc-c++ boost-devel boost
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
OS 166 kB/s | 3.8 kB 00:00
everything 133 kB/s | 3.8 kB 00:00
EPOL 36 kB/s | 3.0 kB 00:00
update 217 kB/s | 3.5 kB 00:00
No match for argument: spdlog
No match for argument: spdlog-devel
Package rpm-4.17.0-22.oe2203sp1.x86_64 is already installed.
Package rpm-build-4.17.0-22.oe2203sp1.x86_64 is already installed.
Package gcc-10.3.1-20.oe2203sp1.x86_64 is already installed.
Package gcc-c+±10.3.1-20.oe2203sp1.x86_64 is already installed.
Error: Unable to find a match: spdlog spdlog-devel
ERROR: Command failed:

/usr/bin/dnf --installroot /var/lib/mock/openeuler-22.03-x86_64/root/ --releasever 22.03 --setopt=deltarpm=False --allowerasing --disableplugin=local --disableplugin=spacewalk --disableplugin=versionlock install spdlog spdlog-devel fmt-devel fmt rpm git procps-ng fuse-devel cmake asciidoc systemd rpm-build zlib-devel gcc gcc-c++ boost-devel boost

Traceback (most recent call last):
File “/usr/bin/copr-sources-custom”, line 88, in
run_cmd(mock + [‘–install’] + pkgs)
File “/usr/bin/copr-sources-custom”, line 64, in run_cmd
return subprocess.check_call(command, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib64/python3.11/subprocess.py”, line 413, in check_call
raise CalledProcessError(retcode, cmd)

2023-03-05 04:55:50 (3.95 MB/s) - ‘epel-release-latest-8.noarch.rpm’ saved [25028/25028]

  • rpm -ivh epel-release-latest-8.noarch.rpm --nodeps
    warning: epel-release-latest-8.noarch.rpm: Header V4 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY
    Verifying… ########################################
    Preparing… ########################################
    Updating / installing…
    epel-release-8-18.el8 ########################################
    Many EPEL packages require the CodeReady Builder (CRB) repository.
    It is recommended that you run /usr/bin/crb enable to enable the CRB repository.
  • yum install -y spdlog spdlog-devel
    OS 136 kB/s | 3.8 kB 00:00
    everything 149 kB/s | 3.8 kB 00:00
    EPOL 107 kB/s | 3.0 kB 00:00
    update 190 kB/s | 3.5 kB 00:00
    No match for argument: spdlog
    No match for argument: spdlog-devel
    Error: Unable to find a match: spdlog spdlog-devel
    Finish: chroot [‘su - mockbuild -c 'set -xe ; cd /workdir ; COPR_OWNER=kerneltravel COPR_PROJECT=lizardfs COPR_PACKAGE='“'”''“'”' COPR_RESULTDIR=/workdir /script'’]
    Traceback (most recent call last):
    File “/usr/bin/copr-sources-custom”, line 115, in
    run_cmd(mock + [‘–chroot’, ‘su - {0} -c {1}’.format(user, cmd)])
    File “/usr/bin/copr-sources-custom”, line 64, in run_cmd
    return subprocess.check_call(command, **kwargs)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “/usr/lib64/python3.11/subprocess.py”, line 413, in check_call
    raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command ‘[‘mock’, ‘-r’, ‘/var/lib/copr-rpmbuild/results/mock-config.cfg’, ‘–chroot’, ‘su - mockbuild -c 'set -xe ; cd /workdir ; COPR_OWNER=kerneltravel COPR_PROJECT=lizardfs COPR_PACKAGE='“'”''“'”' COPR_RESULTDIR=/workdir /script'’]’ returned non-zero exit status 1.

spdlog spdlog-devel 的包,打算暂时用EPEL 提供的包,通过rpm的方式安装上spdlog、spdlog-devel ,

在最新的 EUR 配置中, 设置了 以下包作为前置依赖:

fmt-devel fmt fuse-devel cmake asciidoc systemd wget rpm yum git procps-ng rpm-build zlib-devel gcc gcc-c++ boost-devel boost glibc libstdc++ cmake-filesystem

其中fmt-devel fmt glibc libstdc++ cmake-filesystem 这几个包是 spdlog 和 spdlog-devel 依赖的包,且在openEuler仓库中有,就直接指定自动安装这几个包。

spdlog和spdlog-devel ,通过 rpm 的方式安装:

rpm -ivh --nodeps https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/s/spdlog-1.5.0-2.el8.x86_64.rpm
rpm -ivh --nodeps https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/s/spdlog-devel-1.5.0-2.el8.x86_64.rpm

@mywaaagh_admin EUR 的 build 2803 和 2804 已开始进入代码编译阶段,但都出现了同样的错误日志:如https://eur.openeuler.openatom.cn/results/kerneltravel/lizardfs/srpm-builds/00002803/builder-live.log.gz 目前只能看到 ssh: connect to host 10.0.0.61 port 22: No route to host )

这个情况跟我shell的写法有关吗?

有一些问题,custom方法的目的是生成一个srpm包,EUR会拿着这个SRPM包取构建rpm包,因此你的脚本只需要使用rpmbuild -bs xxx.spec就可以了

1 个赞

跟我之前的理解不一样,现在理解了。

我现在改用SCM方式编译了。src.rpm 生成成功。

https://eur.openeuler.openatom.cn/results/kerneltravel/lizardfs/openeuler-22.03-x86_64/00002814-lizardfs/builder-live.log.gz

这个日志里 遇到新问题:openEuler 22.03 的仓库没有 spdlog 和 spdlog-devel 包,编译liazrdfs中就提示

Copr repository 206 kB/s | 3.3 kB 00:00
OS 235 kB/s | 3.8 kB 00:00
everything 210 kB/s | 3.8 kB 00:00
EPOL 256 kB/s | 3.0 kB 00:00
update 254 kB/s | 3.5 kB 00:00
Package pkgconf-1.8.0-2.oe2203sp1.x86_64 is already installed.
No matching package to install: ‘spdlog’
No matching package to install: ‘spdlog-devel’
Not all dependencies satisfied
Error: Some packages could not be found.

请问 @mywaaagh_admin 像这种情况,EUR有什么办法呢?

openEuler的源里没有spdlog 和 spdlog-devel,你可以再这个project里再添加一个spdlog包,先构建成功spdlog,再构建lizardfs,因为构建包的时候,除了会取官方的源里查找依赖,也会将本项目的源作为依赖

比如:我为了构建i3这个包,将其缺少的依赖包都再i3wm这个仓库里构建了一次

1 个赞

按照老师的指点,现在 spdlog 问题已解决:spdlog在EUR中的package 生成过程:
https://eur.openeuler.openatom.cn/coprs/kerneltravel/lizardfs/build/2821/

spdlog-1.11.0-2.x86_64
systemd-249-46.oe2203sp1.x86_64
unbound-libs-1.13.2-7.oe2203sp1.x86_64
urw-base35-bookman-fonts-20200910-1.oe2203sp1.noarch
urw-base35-c059-fonts-20200910-1.oe2203sp1.noarch
urw-base35-d050000l-fonts-20200910-1.oe2203sp1.noarch
urw-base35-fonts-20200910-1.oe2203sp1.noarch
urw-base35-fonts-common-20200910-1.oe2203sp1.noarch
urw-base35-gothic-fonts-20200910-1.oe2203sp1.noarch
urw-base35-nimbus-mono-ps-fonts-20200910-1.oe2203sp1.noarch
urw-base35-nimbus-roman-fonts-20200910-1.oe2203sp1.noarch
urw-base35-nimbus-sans-fonts-20200910-1.oe2203sp1.noarch
urw-base35-p052-fonts-20200910-1.oe2203sp1.noarch
urw-base35-standard-symbols-ps-fonts-20200910-1.oe2203sp1.noarch
urw-base35-z003-fonts-20200910-1.oe2203sp1.noarch
vim-filesystem-2:9.0-9.oe2203sp1.noarch
wayland-1.20.0-6.oe2203sp1.x86_64
xkeyboard-config-2.33-2.oe2203sp1.noarch
xorg-x11-font-utils-1:7.5-44.oe2203sp1.x86_64
xorg-x11-fonts-7.5-25.oe2203sp1.noarch
zlib-devel-1.2.11-22.oe2203sp1.x86_64

Complete!
Finish: build setup for lizardfs-3.13.0-0.src.rpm
Start: Outputting list of installed packages
Finish: Outputting list of installed packages
Start: rpmbuild lizardfs-3.13.0-0.src.rpm
/etc/profile.d/system-info.sh: line 55: ps: command not found
Building target platforms: x86_64
Building for target x86_64
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.uOAGiN

  • umask 022
  • cd /builddir/build/BUILD
  • cd /builddir/build/BUILD
  • rm -rf lizardfs-3.13.0

继续解决下一个 问题中:

https://eur.openeuler.openatom.cn/results/kerneltravel/lizardfs/openeuler-22.03-x86_64/00002823-lizardfs/builder-live.log

  • /usr/bin/tar -xvvof /builddir/build/SOURCES/lizardfs-3.13.0.tar.gz

gzip: stdin: not in gzip format
/usr/bin/tar: Child returned status 1
/usr/bin/tar: Error is not recoverable: exiting now
error: Bad exit status from /var/tmp/rpm-tmp.uOAGiN (%prep)
Bad exit status from /var/tmp/rpm-tmp.uOAGiN (%prep)

RPM build errors:
Finish: rpmbuild lizardfs-3.13.0-0.src.rpm
Finish: build phase for lizardfs-3.13.0-0.src.rpm
ERROR: Exception(/var/lib/copr-rpmbuild/results/lizardfs-3.13.0-0.src.rpm) Config(openeuler-22.03-x86_64) 0 minutes 42 seconds
INFO: Results and/or logs in: /var/lib/copr-rpmbuild/results
INFO: Cleaning up build root (‘cleanup_on_failure=True’)
Start: clean chroot

这里 提示 /builddir/build/SOURCES/lizardfs-3.13.0.tar.gz 文件 不是gzip 格式。 但我在本地用同样命令能解压得到目录文件:

tar -xvvof lizardfs-3.13.0.tar.gz

不排除EUR网络拉取时的异常,继续处理中……

  • /usr/bin/tar -xvvof /builddir/build/SOURCES/lizardfs-3.13.0.tar.gz

gzip: stdin: not in gzip format
/usr/bin/tar: Child returned status 1
/usr/bin/tar: Error is not recoverable: exiting now
error: Bad exit status from /var/tmp/rpm-tmp.uOAGiN (%prep)
Bad exit status from /var/tmp/rpm-tmp.uOAGiN (%prep)

上述 问题 经验证确实跟网络下载有关。 我原先在rpm/lizardfs.spec 里面用的 gitee.com的文件下载地址下载时response信息不带content-length头的(虽然直接复制 Login - Gitee.com 地址在浏览器能下载,但下载时看不到文件实际大小),所以可能会影响 EUR的curl 或其他工具下载文件的准确性。

换成 正式的 静态文件下载地址 后,上述问题就好了。

请问老师 @mywaaagh_admin

EUR有5小时(Optional - number of seconds we allow the builds to run, default is 18000 (5h))超时的设置。下面这个任务执行了4分钟左右被 terminated 掉了,是因为什么原因呢?

https://eur.openeuler.openatom.cn/results/kerneltravel/lizardfs/openeuler-22.03-x86_64/00002832-lizardfs/builder-live.log

make[3]: Leaving directory ‘/builddir/build/BUILD/lizardfs-3.13.0/build-pack’
[ 20%] Built target manpages
[ 20%] Building CXX object src/common/CMakeFiles/mfscommon_pic.dir/io_limits_database.cc.o
[ 20%] Building CXX object src/common/CMakeFiles/mfscommon_pic.dir/lizardfs_error_codes.cc.o
[ 20%] Building CXX object src/common/CMakeFiles/mfscommon_pic.dir/lockfile.cc.o
[ 20%] Building CXX object src/common/CMakeFiles/mfscommon_pic.dir/loop_watchdog.cc.o
[ 21%] Building CXX object src/common/CMakeFiles/mfscommon_pic.dir/main_options.cc.o
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/protocol/CMakeFiles/lzfsprotocol.dir/build.make:76: src/protocol/CMakeFiles/lzfsprotocol.dir/packet.cc.o] Error 1
make[3]: *** Deleting file ‘src/protocol/CMakeFiles/lzfsprotocol.dir/packet.cc.o’
make[3]: Leaving directory ‘/builddir/build/BUILD/lizardfs-3.13.0/build-pack’
make[2]: *** [CMakeFiles/Makefile2:636: src/protocol/CMakeFiles/lzfsprotocol.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs…
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/common/CMakeFiles/mfscommon_pic.dir/build.make:76: src/common/CMakeFiles/mfscommon_pic.dir/access_control_list.cc.o] Error 1
make[3]: *** Deleting file ‘src/common/CMakeFiles/mfscommon_pic.dir/access_control_list.cc.o’
make[3]: *** Waiting for unfinished jobs…
[ 21%] Building CXX object src/common/CMakeFiles/mfscommon.dir/connection_pool.cc.o
[ 21%] Building CXX object src/common/CMakeFiles/mfscommon.dir/crc.cc.o
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/common/CMakeFiles/mfscommon_pic.dir/build.make:188: src/common/CMakeFiles/mfscommon_pic.dir/connection_pool.cc.o] Error 1
make[3]: *** Deleting file ‘src/common/CMakeFiles/mfscommon_pic.dir/connection_pool.cc.o’
[ 21%] Building CXX object src/common/CMakeFiles/mfscommon.dir/cwrap.cc.o
[ 22%] Building CXX object src/common/CMakeFiles/mfscommon.dir/disk_info.cc.o
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/common/CMakeFiles/mfscommon_pic.dir/build.make:174: src/common/CMakeFiles/mfscommon_pic.dir/chunkserver_stats.cc.o] Error 1
make[3]: *** Deleting file ‘src/common/CMakeFiles/mfscommon_pic.dir/chunkserver_stats.cc.o’
[ 22%] Building CXX object src/common/CMakeFiles/mfscommon.dir/event_loop.cc.o
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/common/CMakeFiles/mfscommon.dir/build.make:76: src/common/CMakeFiles/mfscommon.dir/access_control_list.cc.o] Error 1
make[3]: *** Deleting file ‘src/common/CMakeFiles/mfscommon.dir/access_control_list.cc.o’
make[3]: *** Waiting for unfinished jobs…
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/common/CMakeFiles/mfscommon_pic.dir/build.make:146: src/common/CMakeFiles/mfscommon_pic.dir/chunk_connector.cc.o] Error 1
make[3]: *** Deleting file ‘src/common/CMakeFiles/mfscommon_pic.dir/chunk_connector.cc.o’
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/common/CMakeFiles/mfscommon_pic.dir/build.make:216: src/common/CMakeFiles/mfscommon_pic.dir/cwrap.cc.o] Error 1
make[3]: *** Deleting file ‘src/common/CMakeFiles/mfscommon_pic.dir/cwrap.cc.o’
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[3]: *** [src/uraft/CMakeFiles/lizardfs-uraft.dir/build.make:76: src/uraft/CMakeFiles/lizardfs-uraft.dir/main.cc.o] Error 1
make[3]: *** Deleting file ‘src/uraft/CMakeFiles/lizardfs-uraft.dir/main.cc.o’
make[3]: *** Waiting for unfinished jobs…

看到社区其他组提到过类似的一个问题:

可以等会儿再试试还有没有问题,下午运维同学更新了builder镜像,下次我们尽量吧SOP维护放到晚些时候,不好意思

可能跟 编译的默认线程数和占用内存 超过了EUR承受能力 有关?
3月7日上午7:40左右开始的编译仍然出现compilation terminated 的情况。

https://eur.openeuler.openatom.cn/results/kerneltravel/lizardfs/openeuler-22.03-x86_64/00002935-lizardfs/builder-live.log

有可能是触发了oom-killer,试试吧spec里的make %{?_smp_mflags},改成make -j4再试试呢

1 个赞

改成 make -j4 以后,Lizardfs 3.13.0(最新版)已经编译成功了。

https://eur.openeuler.openatom.cn/coprs/kerneltravel/lizardfs/build/2938/

https://eur.openeuler.openatom.cn/results/kerneltravel/lizardfs/openeuler-22.03-x86_64/00002938-lizardfs/

:kissing_heart: 感谢 @mywaaagh_admin 答疑指点!!!

接下来要对编译的配置文件做一些 规范化。因为之前打包中注释掉了spec里面的mfsmount3这个最新功能,以及修改了src的 URL等地址。如果按照openEuler的正式打包要求,会需要向lizardfs上游进行代码仓库合并,然后使用上游的仓库进行打包。

1 个赞