28 KiB

Raw Permalink Blame History

openKylin source code independent selection build process

I. Package version number selection

1. selection strategy

1.1 Software project selection strategy

Continue to actively introduce full "A" software (version)
Encourage the introduction of "A" and "B" software only (version) tory-view-113483.html
Cautiously introduce software with "C" (version)
Reject/delete software (version) with "D" or "C >= 3" in time

1.2 Software version selection strategy

Long-term support version (for the time being, openKylin does not have a long-term support version, 1.0 is not a long-term support version at the moment)
- Give priority to the version widely used by mainstream OS, taking into account both quality and stability.
- The compatibility list is not affected by the upgrade.
- No major version changes during the life cycle of the same version, only feature and patch rounds to solve quality problems.
Community development version
- Meet the basic compliance issues, integrate as much software as possible, giving priority to the latest version of the current stable branch of open source components.
- Follow up on upstream community dynamics in a timely manner and merge new feature versions of open source software.

1.3 Software classification and compatibility principles

Level	Scope	Principles
Level 1	kernel, glibc, gcc, zlib, and other core system components	APIs and ABI remain stable for the lifetime of the main release, and as much as possible for the next main release
Level 2	Base software such as dbus, oepnssl, bzip2, systemd, etc.	APIs and ABIs remain stable for the lifetime of a single major release, and dependent applications do not require major changes in principle for the lifetime of a single major release
The API and ABI are not forced to remain stable during the life cycle of a single major version, and dependent applications are upgraded inline.

The specific process

select the new projects to be introduced according to the "Software Project Selection Policy". 2.
select a version of the project according to the "software version selection policy"
output the "project selection report", including the compliance, compatibility, importance, activity, quality, security details of the new project, as well as the new features of the selected version, compatibility, impact domain, comparison of versions in other distributions, etc.
the technical committee adopted the following process to carry out specific work

2.1 Obtain the project address of the package

The project address is used to view the current project details, development status, compilation and installation, instructions, etc. of the package

There are several ways to obtain the project address of the package.

2.1.1 The debian community's package tracking platform

Most packages can be found on the debian community's package tracker (tracker.debian.org)

For example, to find the libnftnl package

The project address of the package can be accessed through the homepage information in the upper right corner, as shown above.

2.1.2 dsc file

The dsc file of a package can be viewed on the debian community (tracker.debian.org), ubuntu (https://launchpad.net), etc. The dsc file may contain information describing the project address of the current package .

For example: libnftnl in the debian community to view the package dsc file (hereinafter in version 1.1.2-2) debian/pool/main/libn/libnftnl/libnftnl_1.1.2-2.dsc) as an example).

# Remove gpg signature information for easy viewing
Format: 3.0 (quilt)
Source: libnftnl
Binary: libnftnl11, libnftnl-dev
Architecture: linux-any
Version: 1.1.2-2
Maintainer: Debian Netfilter Packaging Team <pkg-netfilter-team@lists.alioth.debian.org>
Uploaders: Arturo Borrero Gonzalez <arturo@debian.org>
Homepage: https://git.netfilter.org/libnftnl
Standards-Version: 4.2.1
Vcs-Browser: https://salsa.debian.org/pkg-netfilter-team/pkg-libnftnl
Vcs-Git: https://salsa.debian.org/pkg-netfilter-team/pkg-libnftnl
Build-Depends: debhelper (>= 11), libmnl-dev, libtool, pkg-config
package-list:
 libnftnl-dev deb libdevel optional arch=linux-any
 libnftnl11 deb libs optional arch=linux-any
Checksums-Sha1:
 c0f880588fabaa845a88fb5a2bdad0dea7481857 366014 libnftnl_1.1.2.orig.tar.bz2
 0a9063e79191955fbf59482d75c7bfb62aded816 912 libnftnl_1.1.2.orig.tar.bz2.asc
 505f8b44e3adb574dc3cf657580e9d73f0b0a853 8772 libnftnl_1.1.2-2.debian.tar.xz
Checksums-Sha256:
 a5c7b7a6c13c9c5898b13fcb1126fefce2015d5a96d7c354b19aaa40b6aece5d 366014 libnftnl_1.1.2.orig.tar.bz2
 4178552d7ad210e9f17df3c079ac6de032f62c72774d59bfac404a023d48950b 912 libnftnl_1.1.2.orig.tar.bz2.asc
 4ef80a0c344b23624d24b7dd4be1b73a7b96d86089813c545705f8093067ce13 8772 libnftnl_1.1.2-2.debian.tar.xz
Files:
 14093a238d5025d4a452e6d1cef88c58 366014 libnftnl_1.1.2.orig.tar.bz2
 e10dcb9138261d92e7569bd72b809a28 912 libnftnl_1.1.2.orig.tar.bz2.asc
 518ff0360f67c8031a31792a257a5c3b 8772 libnftnl_1.1.2-2.debian.tar.xz

As highlighted in.

Homepage : project address
Vcs-Browser : Version control system browsing platform (similar to github, gitee, gitlab, etc.)
Vcs-Git : git repository address

2.1.3 debian/control files

First download any source package from debian community, ubuntu community (launchpad.net) and unzip it.

# Download the source package from the ubuntu community (launchpad.net)
# You can also use apt-source to download source code by adding deb-src source
# To use dget you need to install devscripts
dget -x https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/libnftnl/1.2.3-1/libnftnl_1.2.3-1.dsc

# Go to the unpacked directory
cd libnftnl-1.2.3

Check the debian/control file for Homepage, Vcs-Git, Vcs-Browser and other related fields

For example.

``Bash $ grep -E "^(Homepage|Vcs-*)" debian/control Homepage: https://git.netfilter.org/libnftnl Vcs-Git: https://salsa.debian.org/pkg-netfilter-team/pkg-libnftnl.git Vcs-Browser: https://salsa.debian.org/pkg-netfilter-team/pkg-libnftnl


The descriptions of the fields highlighted above are the same as in subsection 2.1.2

#### 2.1.4 debian/watch files

As in subsection 2.1.3, the source code needs to be downloaded first. This time, the `debian/watch` file is viewed

The **debian/watch** file is mainly used to record the download address of the upstream source code (this file is not required, so it may not be present in some source packages)

``Bash
$ cat debian/watch  
version=3
opts=pgpsigurlmangle=s/$/.sig/ http://ftp.netfilter.org/pub/libnftnl/libnftnl-(\S+).tar.bz2

The upstream source code download address recorded in this file allows you to find its corresponding project address

For example.

# The download address is usually an ftp or git release download address
# If the download address is hosted by another platform, then you may not know the project address
http://ftp.netfilter.org/pub/libnftnl

# The above address is a good guess for the project address
http://netfilter.org/pub/libnftnl

2.2 Getting the package version status

The latest stable version list of the current project is learned by looking at the project address.

This can be viewed in several ways.

debian/watch file

If the package in subsection 2.1.4 has a debian/watch file, you can view the download address of the upstream package zip of the project.

! The download address may contain a beta version with a name similar to xxxx-beta.tar.gz. Generally this beta version of the source code should not be selected.
```!

- git repositories

If it is a git repository, you can check if its counterpart has a release git release: 1.

1. If there is, then you can get a list of its releases on the git release page

2. If not, see if the corresponding `branch(branch)` or `tag(tag)` has the relevant version list information



# II. Building the source package

Select the version of the package to be built and the download address from the first section.

If there is already a project in the remote repository and you need to change the upstream version, then please check the contents of subsection 4.

## 1. Download the source code and modify the source code directory format

Download the source package locally and unzip it

```Bash
## Download the source code
wget https://www.netfilter.org/pub/libnftnl/libnftnl-1.2.3.tar.bz2

# Decompress
tar -xaf libnftnl-1.2.3.tar.bz2

Modify the source directory to conform to the <source_name>-<source_version> format. For example.

$ ls -al
drwxr-x--@ - luzp 10 Aug 02:24 libnftnl-1.2.3
.rw-rw----@ 395k luzp 10 Aug 02:24 libnftnl-1.2.3.tar.bz2

2. build out the debian package directory

2.1 upstream source contains debian directory

! Note: If the upstream source contains a debian directory, we need to manually rename the upstream debian directory to avoid other problems when we regenerate the debian directory

Back up the original debian directory

mv debian debian-orig

Rebuild the debian directory

debmake -u <upstream version number> -r <revision number> -t

This will create a new directory at the same level as the source code, in the format -

Suppose.

The source tarball is amdgcn-tools-13.tar.gz

``Bash

Extract the source code

Unzip the source code to the amdgcn-tools-13 directory

tar -xaf amdgcn-tools-13.tar.gz

Go to the source directory and backup the original debian directory

cd amdgcn-tools-13 mv debian debian-orig

Rebuild the debian directory

debmake -u13 -rok1 -t

After execution, a new directory will be created in the same level as amdgcn-tools-13

-, which is the following directory

ls amdgcn-tools-13-13

### 2.2 Building debian directories directly

If the upstream source code is pure source, then just build the debian directory directly.

``Bash
### Build the debian directory
debmake -z tar.bz2

-z Specify the tarball format for the source code

libnftnl-1.2.3.tar.bz2 would be specified as -z tar.bz2
The default source compression format is tar.gz

-b If it is a perl or python source package, you can specify this parameter to create more files for debmake in the corresponding programming language

# If it is python
debmake -b ":python3"

# perl
debmake -b ":perl"

-t regenerate the upstream source package (i.e. xxx.orig.tar.gz)

More detailed debmake usage can be found in its man page

man debmake

3. Modify build rules

The main thing is to modify the files in the debian directory.

3.1 control

The control file covers the contents of two sections.

3.1.1 Source package section

A brief description of the current source package, and the compilation dependencies for building it

There are several fields that need to be noted.

Section

Describes the type of package, and is used to classify the package. For example, mysql-8 should belong to database.

The options are.

admin, cli-mono, comm, database, debug, devel, doc, editors, education, electronics, embedded, fonts, games, gnome, gnu-r, gnustep, graphics, hamradio, haskell, httpd, interpreters, introspection, java, javascript, kde, kernel, libdevel, libs, lisp, localization, mail, math, metapackages, misc, net, news, ocaml, oldlibs, otherosfs, perl, php, python, ruby, rust, science, shells, sound, tasks, tex, text, utils, vcs, video, web, x11, xfce, zope

Priority
- Priority of the package
- required System required packages
- important Important packages, packages that are not required for the system, but preferably pre-installed on the system
- standard Standard Relatively independent packages
- optional This is the default priority option, which should be used unless the package is a system-required package
Maintainer It needs to be written as the current openkylin sig group responsible for the package, e.g. packaging sig group is written as Openkylin Developers <packaging@lists.openkylin.top>
Vcs-Browser:

Write it to the corresponding gitee repository address, e.g. https://gitee.com/openkylin/autoconf

Vcs-Git.

Write to the corresponding gitee repository git address, e.g. https://gitee.com/openkylin/autoconf.git

**Build-Depends and **

Build dependencies needed to compile architecture-related code

Build-Depends-Indep

Compilation dependencies required to compile architecture-independent code

For Build-Depends and Build-Depends-Indep you can find the relevant compilation dependencies from the project address.

Note: If the control file references rules from another OS distribution, you need to remove or replace the version number rules in the other OS version, for example.

Build-Depends: 
  pkg1 (>= 1.0.0-0ubuntu1), 
  pkg1 (< 1.2.0-1debian2),
  pkg2 (> 1.0.0.build),
  pkg3 (> 2.0ubuntu19)

First of all, it should be clear that for quilt format version numbers generally have "-" in the format <upstream_version>-<revision>, while native format source packages generally do not contain "-"

The modification process can be done in two steps

Step 1:

For quilt-formatted versions, the version numbers 1.0.0-0ubuntu1 and 1.2.0-1debian2 in pkg1 contain the version number of the specific OS distribution, so they should be modified. It also becomes.

pkg1 (>= 1.0.0), 
pkg1 (< 1.2.0),

For version numbers in native format, you first need to determine its upstream version, for pkg2 and pkg3.

(1) Try to find the 1.0.0 version of pkg2, if you can find it, then 1.0.0 is the upstream version, and its compilation dependency should be changed to: pkg2 (> 1.0.0); same as pkg3

(2) If the upstream version of pkg2 cannot be found, and the current version number does not contain the name of other OS distributions, then it can be kept (because the source-package tool is used to repackage the source code, its upstream version number is also the current version number); but for pkg3, if pkg3 is OS self-developed (its source code name usually However, for pkg3, if pkg3 is OS developed (its source code name also contains OS name, e.g. ubuntu-keyring), then this package can be renamed and maintained by openkylin, with the corresponding source code name changed to openkylin-keyring and the version number redefined, e.g. 1:1.0.0; if it is not OS developed package, it is better for us to redefine the version number and maintain it according to If it is not an OS self-developed package, it is better that we also re-define the version number and maintain it according to our own version number, which can be discussed internally, e.g. 1:1.0.0 is also specified here.** Note that here, it is better to use the version number in : format, which indicates that the version number with : is always higher than the version number without. **

Then, for pkg2, pkg3, it is modified as follows

pkg2 (> 1.0.0.build),
pkg3 (> 1:1.0.0)

Step 2 (native format is not handled):

The second step is to improve on the first step. In general we need to clarify the content of the revisions in other OS distributions that follow the - horizontal line.

For <upstream_version>-, generally each revision is a fix for a bug, security vulnerability, etc. For ubuntu, each revision is made up of several patches.

So for pkg1 (>= 1.0.0-0ubuntu1), it is explicitly stated that it can only be used after revision 0ubuntu1, which means that using pkg1 version 1.0.0 is risky, and the most immediate phenomenon is that the compilation does not pass.

Therefore, it is necessary to combine our actual pkg1 version situation to.

(1) If pkg1 exists in the repository, its version is 1.0.0-ok1. In order to ensure that the above compilation passes, it is necessary to integrate the patch in ubuntu's 1.0.0-0ubuntu1. If the relevant patch is already integrated, then the compilation dependency becomes.

pkg1 (>= 1.0.0-ok1), 
pkg1 (< 1.2.0),

After the patch integration, the pkg1 version in the repository becomes 1.0.0-ok2. Then, where the dependencies are compiled, it becomes

pkg1 (>= 1.0.0-ok2), 
pkg1 (< 1.2.0),

(2) If pkg1 does not exist in the repository, then when we select the build independently for the first time, it is better to select the version between 1.0.0~1.2.0 to ensure that the compiled dependencies of each dependency package can be met.

The above is based on compilation dependencies. If the version number is also specified in the installation dependency field of the binary package, then it is handled in the same way as above.

3.1.2 Binary package section

Describes which binary package programs the current source package can be divided into.

Common types of subpackages are

shared library, whose binary package name is usually lib-xxx.

Shared library headers and related development files, usually named lib-xxx-dev

Documentation of a package

If the documentation corresponds to the documentation of an executable, the binary package name is usually xxx-doc.

If the documentation is for a development library, the binary package name is usually lib-xxx-doc.

The binary package name is usually the name of the corresponding binary program of a package, such as the binary program of a compiled language (e.g., c/c++) and the executable program of an interpreted programming language (e.g., python, perl)

This sub-package can be used in combination with debmake when building the ``debian` package directory, e.g.

# The package name is foo, which provides the binary and the documentation for the binary
debmake -b "foo:bin,foo-doc:doc"

# The package name is libfoo, which is the source code for a development library that provides documentation for the development library and the development library
debmake -b "libfoo:lib,libfoo-doc:doc"

# After execution, each binary package information template will be automatically written to debian/control file

3.2 rules

Package compilation and build rules

3.3 changelog

Modify the package version number in the format: <upstream_version>-

In general, for the openkylin community, use ok1 for the first revisions, e.g.

autoconf (2.71-ok1) yangtze; urgency=low

  * Build for openKylin.

 -- Lu zhiping <luzhiping@kylinos.cn> Fri, 26 Aug 2022 15:25:09 +0800

3.4 source/format

The source package must be formatted as quilt format

echo "3.0 (quilt)" > debian/source/format

echo "3.0 (quilt)" > debian/source/format

3.5 Other files

xxx.install

xxx.doc

xxx.symbols

xxx.manpages

3.6 Applying patches from other OS distributions

If other OS distributions have patches for the package, then we need to analyze and consider whether to apply their patches.

You can start by considering whether to apply patches from other OS distributions in the following ways.

What is fixed by the patch? Is it an irrelevant patch that has not been absorbed by the upstream community?
Is it a patch that was only available in a previous, older release and has it been integrated in a new release?
Is it a major vulnerability fix patch?
Is it a patch that is only required for a specific OS distribution?
If the patch is not applied is there a compilation problem and is there a problem with the installation?

Also, it's best to apply patches from the upstream community after building out the git repository, so that the information from the upstream community patches can be retained in the git commit

Here are the steps for applying patches after building out a git repository in Section 5.

If you need to apply patches from upstream communities, it is best to keep the content and author information of their patches. You can apply patches from upstream communities directly through the git tool.

git am apply patch The git am application patch will automatically keep the patch information

``Bash git am xxxx.patch


- git apply applies patch

But some patches don't have the exact same information format as git am, so you can use git apply to import the patches and then commit to add the patch information

```Bash
# import patch information
git apply xxx.patch

# Add changes
git add .

# Fill in the edit box with information about the patch, such as
# the author of the patch, the content of the patch, etc.
git commit
git commit .

After importing a patch, if you want to compile and test the newly imported patch, you can use the gbp tool to generate the new source code (new dsc file)

```Bash
# Modify changelog to add a new version number
vim debian/changelog

# Compile the source code, which will generate a new dsc file
gbp buildpackage -S -sa -nc --git-prisine-tar --git-ignore-branch

4. Importing upstream source code

If the remote repository already has a project, you need to change the upstream version.

(1) Download the source tarball from the upstream community

(2) Re-compress the source tarball to match the ``_.orig.tar.[gz|xz|bz2]` format

# For example.
# The source tarball obtained from the upstream source community is requests-2.27.1.tar.gz
# First extract the source code
tar -xaf requests-2.27.1.tar.gz

# Change the source directory to <package-name>-<package-version>
# If the source tarball from the upstream community is extracted from the requests directory
mv requests requests-2.27.1

# Recompress to match <package-name>_<pacakge-version>.orig.tar.[gz|xz|bz2] 
tar -czf requests-2.27.1 requests-2.27.1.orig.tar.gz

(3) Clone the gitee remote repository source code to local, enter the source code directory

# Clone the remote repository
git clone git@gitee.com:openkylin/requests.git

# Enter the source directory
cd requests

(4) Changing upstream source code

Refer to chapter 6 of the [openKylin source package git workflow](openKylin source package git workflow.md) section "Updating upstream versions".

a. Import the upstream version (the zip generated in step 2)
``Bash gbp import-orig requests-2.27.1.orig.tar.gz --no-merge --pristine-tar --upstream-branch=upstream git checkout upstream git push git push --tags
b. Filtering old patches (in conjunction with [openKylin source package git workflow] (openKylin source package git workflow.md) subsection 6.3.3)
If the patch is an R&D patch, then you need to confirm with the relevant R&D that the patch is still required in the new upstream source code
- If the patch is a patch from another OS distribution, then confirm with the OS distribution that the patch is still required in the latest version of the source code
c. Introducing patches from other OS distributions
You can refer to subsection 3.6, "Applying patches from other OS distributions".
- Use git am or git apply or other methods to import patches from other OS distributions
d. Modify other build files in the debian directory
debian/control
- debian/changelog needs to retain the original changelog information
- debian/rules
- deiban/other files

After the changes are done, you can follow section 5 and test the build locally first, then push it to the remote repository after the test is done.

5. Compile the source package

After completing the build rules for the debian directory, first compile a round of source packages based on this. The dsc file will be generated

``Bash

Switch to the development branch

git checkout openkylin/yangtze

Execute in the source directory

dpkg-source -b .




# III. Compile

After building the source package according to the previous section, compile it.

You can download a chroot environment of openkylin and compile and debug in the chroot environment.

The chroot for x86: http://files.build.openkylin.top/51159/chroot-openkylin-yangtze-amd64.tar.gz

riscv chroot: http://files.build.openkylin.top/51197/chroot-openkylin-yangtze-riscv64.tar.gz

At this stage, the package build rules in the debian package directory will be continuously modified and optimized until a complete installable binary package is packaged.



# IV. Installation Testing

Simply test whether the compiled binary is installable and whether the system works properly after installation, etc.

## V. Building a git repository

## 1. create gitee repository

fork openkylin's community repository

``Bash
# After fork to your personal repository, clone your personal repository to local
git clone git@gitee.com:<username>/community.git

# Then add the source package name to the corresponding sig group sig.yaml file
# For example, to add a package to the packaging sig, the source-name should be the actual package name
echo "- <source-name>" >> sig/packaging/sig.yaml

# Then commit the changes and push them to your personal repository
git add sig/packaging/sig.yaml
git commit -m "Add xxx package"
git push origin master

Create a PR on the gitee web side, and then have it reviewed by the community project leader. After the review, the CI platform will automatically create the corresponding git repository

2. Build the git repository

Build the git repository for the source package built in chapter 2.

Download the tool

Follow the source-packaging tool instructions to download the tools and install the dependencies.

Build the git repository

. /source-packing import-to-git --dsc-file sqlite3_3.31.1-ok1.dsc --packaging-branch openkylin/yangtze

More detailed git workflow can be found in [openKylin source package git workflow](openKylin source package git workflow.md)

3. Associating remote repositories and pushing to gitee

You need the package to be compiled in chapter 3 and installed and tested in chapter 4 before pushing it to the gitee repository

# Go to the source code repository built in step 2
cd sqlite3
# Associate the remote repository and push to it
git remote add origin git@gitee.com:openkylin/sqlte3.git
git push --all && git push --tags

VI. Other content

A brief description of gitee's ci process.

1. package first push.
maintainer first pushes the package to the gitee repository by way of push, at which point the ci platform sends its package to the okbs compilation platform for compilation, and the current compilation address is (https://build.openkylin.top/~cibot/+archive/openkylin/ citest/+packages)

2. Subsequent modification contributions.
The developer needs to fork the repository to the personal repository, then develop changes on the personal repository, and then modify the changelog to add the version number after the source code is modified, and then submit the PR to the original repository.

maintainer review PR, PR review passed, CI platform automatically package the current PR sent to OKBS compilation platform for compilation, the compilation results will be added to the corresponding PR in the form of comments, maintainer according to the compilation results and CI platform gate check results to consider whether to accept the PR

The source address of each distribution, using gvfs as an example.

Debian 11: https://tracker.debian.org/pkg/gvfs

Ubuntu 22.04: https://launchpad.net/ubuntu/+source/gvfs

Fedora 36: https://src.fedoraproject.org/rpms/gvfs

Arch: https://archlinux.org/packages/extra/x86_64/gvfs/

Deepin: https://ftp.sjtu.edu.cn/deepin/pool/

openEuler: https://gitee.com/src-openeuler/gvfs

28 KiB Raw Permalink Blame History