Skip to content

Commit

Permalink
create-kernel-deps-specs-from-template.sh: create specs based on temp…
Browse files Browse the repository at this point in the history 
…late

Move the existing kernel dep specs to .in template & generate spec files
at the run time for each kernel version supported.

Patch package builder to consider version+release while building
packages.

Change-Id: Ifa9a47eeabe9bf9a11ac576fa5ae92a2b401d33e
Signed-off-by: Shreenidhi Shedi <shreenidhi.shedi@broadcom.com>
Reviewed-on: http://photon-jenkins.eng.vmware.com:8082/c/photon/+/23439
Reviewed-by: Alexey Makhalov <amakhalov@vmware.com>
Tested-by: gerrit-photon <photon-checkins@vmware.com>
  • Loading branch information
@sshedi
sshedi committed Mar 22, 2024
1 parent 83be61d commit 21bd6f2
Show file tree
Hide file tree
Showing 12 changed files with 148 additions and 62 deletions.
3 changes: 3 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ output-*/
tools/bin/
tools/src/contain/contain_unpriv
common/data/pkg_info.json
SPECS/falco/*.spec
SPECS/sysdig/*.spec
SPECS/linux/linux-rt-drivers-intel*.spec
26 changes: 15 additions & 11 deletions SPECS/falco/falco.spec → SPECS/falco/falco.spec.in
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
%global security_hardening none

%define uname_r %{KERNEL_VERSION}-%{KERNEL_RELEASE}

%define _modulesdir /lib/modules/%{uname_r}

Summary: The Behavioral Activity Monitor With Container Support
Name: falco
Version: 0.36.2
Release: 2%{?kernelsubrelease}%{?dist}
Release: 3%{?kernelsubrelease}%{?dist}
License: GPLv2
URL: https://falco.org
Group: Applications/System
Expand Down Expand Up @@ -94,6 +92,7 @@ export KERNELDIR="%{_modulesdir}/build"
%{cmake_install}
mkdir -p %{buildroot}%{_modulesdir}/extra
install -vm 644 %{__cmake_builddir}/driver/%{name}.ko %{buildroot}%{_modulesdir}/extra
find %{buildroot}%{_modulesdir} -name *.ko -type f -print0 | xargs -0 xz

%clean
rm -rf %{buildroot}/*
Expand All @@ -107,10 +106,11 @@ rm -rf %{buildroot}/*
%files
%defattr(-,root,root)
%{_bindir}/*
%{_usrsrc}/*
%exclude %{_usrsrc}/debug
%{_usrsrc}/%{name}*
%{_sysconfdir}/%{name}
%{_datadir}/%{name}
%{_modulesdir}/extra/%{name}.ko
%{_modulesdir}/extra/%{name}.ko.xz
%{_sysconfdir}/falcoctl/falcoctl.yaml

%files devel
Expand All @@ -120,17 +120,21 @@ rm -rf %{buildroot}/*
%{_includedir}/falcosecurity/*

%changelog
* Tue Jan 09 2024 Ankit Jain <ankitja@vmware.com> 0.36.2-3
- compress .ko and exclude debug from main package
* Thu Dec 14 2023 Piyush Gupta <gpiyush@vmware.com> 0.36.2-2
- Bump up version to compile with new go
* Thu Nov 30 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.36.2-1
- Upgrade to v0.36.2
* Sun Nov 19 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.32.2-11
- Bump version as a part of openssl upgrade
* Wed Oct 11 2023 Piyush Gupta <gpiyush@vmware.com> 0.32.2-10
* Wed Nov 29 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.32.2-12
- Bump version as a part of protobuf upgrade
* Wed Oct 11 2023 Piyush Gupta <gpiyush@vmware.com> 0.32.2-11
- Bump up version to compile with new go
* Mon Sep 18 2023 Piyush Gupta <gpiyush@vmware.com> 0.32.2-9
* Mon Sep 18 2023 Piyush Gupta <gpiyush@vmware.com> 0.32.2-10
- Bump up version to compile with new go
* Wed Aug 09 2023 Mukul Sikka <msikka@vmware.com> 0.32.2-8
* Wed Aug 23 2023 Mukul Sikka <msikka@vmware.com> 0.32.2-9
- Bump version as a part of grpc upgrade
* Mon Jul 31 2023 Mukul Sikka <msikka@vmware.com> 0.32.2-8
- Bump version as a part of grpc upgrade
* Mon Jul 17 2023 Piyush Gupta <gpiyush@vmware.com> 0.32.2-7
- Bump up version to compile with new go
Expand All @@ -140,7 +144,7 @@ rm -rf %{buildroot}/*
- Bump version as a part of lua upgrade
* Sat Jun 17 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.32.2-4
- Bump version as a part of protobuf upgrade
* Fri Jun 09 2023 Nitesh Kumar <kunitesh@vmware.com> 0.32.2-3
* Thu Jun 01 2023 Nitesh Kumar <kunitesh@vmware.com> 0.32.2-3
- Bump version as a part of ncurses upgrade to v6.4
* Fri Apr 14 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.32.2-2
- Bump version as a part of zlib upgrade
Expand Down
18 changes: 11 additions & 7 deletions SPECS/sysdig/sysdig.spec → SPECS/sysdig/sysdig.spec.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
Summary: Sysdig is a universal system visibility tool with native support for containers.
Name: sysdig
Version: 0.34.1
Release: 1%{?kernelsubrelease}%{?dist}
Release: 2%{?kernelsubrelease}%{?dist}
License: GPLv2
URL: http://www.sysdig.org
Group: Applications/System
Expand Down Expand Up @@ -117,6 +117,7 @@ rm -rf %{buildroot}%{_datadir}/zsh/ \

mkdir -p %{buildroot}%{_modulesdir}/extra
mv %{__cmake_builddir}/driver/scap.ko %{buildroot}%{_modulesdir}/extra
find %{buildroot}%{_modulesdir} -name *.ko -type f -print0 | xargs -0 xz

%clean
rm -rf %{buildroot}/*
Expand All @@ -130,22 +131,25 @@ rm -rf %{buildroot}/*
%files
%defattr(-,root,root)
%{_bindir}/*
%{_usrsrc}/*
%exclude %{_usrsrc}/debug
%{_usrsrc}/scap*
%{_datadir}/%{name}/*
%{_datadir}/bash-completion/*
%{_mandir}/*
%{_modulesdir}/extra/scap.ko
%{_modulesdir}/extra/scap.ko.xz

%changelog
* Tue Jan 09 2024 Ankit Jain <ankitja@vmware.com> 0.34.1-2
- compress .ko and exclude debug from main package
* Wed Nov 29 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.34.1-1
- Upgrade to v0.34.1
* Sun Nov 19 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.30.2-7
- Bump version as a part of openssl upgrade
* Wed Aug 09 2023 Mukul Sikka <msikka@vmware.com> 0.30.2-6
* Wed Aug 23 2023 Mukul Sikka <msikka@vmware.com> 0.30.2-7
- Bump version as a part of grpc upgrade
* Mon Jul 31 2023 Mukul Sikka <msikka@vmware.com> 0.30.2-6
- Bump version as a part of grpc upgrade
* Sat Jun 17 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.30.2-5
- Bump version as a part of protobuf upgrade
* Fri Jun 09 2023 Nitesh Kumar <kunitesh@vmware.com> 0.30.2-4
* Thu Jun 01 2023 Nitesh Kumar <kunitesh@vmware.com> 0.30.2-4
- Bump version as a part of ncurses upgrade to v6.4
* Fri Apr 14 2023 Shreenidhi Shedi <sshedi@vmware.com> 0.30.2-3
- Bump version as a part of zlib upgrade
Expand Down
2 changes: 2 additions & 0 deletions build.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -680,6 +680,8 @@ def __init__(self):

CheckTools.check_spec_files()

runBashCmd("./tools/scripts/generate-specs-from-templates.sh")

Utilities(None).generate_dep_lists()

if constants.buildArch != constants.targetArch:
Expand Down
7 changes: 4 additions & 3 deletions support/package-builder/PackageBuilder.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -183,9 +183,10 @@ def _installDependencies(self, arch, deps=[]):
def _buildPackagePrepareFunction(self, package, version, doneList):
self.package = package
self.version = version
self.logName = f"build-{package}-{version}"
pkg = f"{package}-{version}"
self.logName = f"build-{pkg}"
self.logPath = (
f"{constants.logPath}/{package}-{version}.{constants.currentArch}"
f"{constants.logPath}/{pkg}.{constants.currentArch}"
)
if not os.path.isdir(self.logPath):
self.cmdUtils.runBashCmd(f"mkdir -p {self.logPath}")
Expand Down Expand Up @@ -272,7 +273,7 @@ def _installPackage(
)
raise Exception("Missing rpm file")
specificRPM = os.path.basename(rpmfile.replace(".rpm", ""))
pkg = package + "-" + packageVersion
pkg = self._findPackageNameAndVersionFromRPMFile(f"{package}-{packageVersion}")
if pkg in listInstalledPackages:
return
# mark it as installed - to avoid recursion
Expand Down
9 changes: 3 additions & 6 deletions support/package-builder/PackageUtils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -225,9 +225,8 @@ def findRPMFile(self, package, version="*", arch=None, throw=False):

if version == "*":
version = SPECS.getData(arch).getHighestVersion(package)
release = SPECS.getData(arch).getRelease(package, version)
buildarch = SPECS.getData(arch).getBuildArch(package, version)
filename = f"{package}-{version}-{release}.{buildarch}.rpm"
filename = f"{package}-{version}.{buildarch}.rpm"

fullpath = f"{constants.rpmPath}/{buildarch}/{filename}"
if os.path.isfile(fullpath):
Expand All @@ -245,8 +244,7 @@ def findRPMFile(self, package, version="*", arch=None, throw=False):
def findSourceRPMFile(self, package, version="*"):
if version == "*":
version = SPECS.getData().getHighestVersion(package)
release = SPECS.getData().getRelease(package, version)
filename = f"{package}-{version}-{release}.src.rpm"
filename = f"{package}-{version}.src.rpm"

fullpath = f"{constants.sourceRpmPath}/{filename}"
if os.path.isfile(fullpath):
Expand All @@ -259,8 +257,7 @@ def findDebugRPMFile(self, package, version="*", arch=None):

if version == "*":
version = SPECS.getData(arch).getHighestVersion(package)
release = SPECS.getData(arch).getRelease(package, version)
filename = f"{package}-debuginfo-{version}-{release}.{arch}.rpm"
filename = f"{package}-debuginfo-{version}.{arch}.rpm"

fullpath = f"{constants.rpmPath}/{arch}/{filename}"
if os.path.isfile(fullpath):
Expand Down
41 changes: 16 additions & 25 deletions support/package-builder/SpecData.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def _readSpecs(self, specFilesPath):

# skip the specfile if buildarch differs
buildarch = spec.packages.get("default").buildarch
if buildarch != "noarch" and buildarch != self.arch:
if buildarch not in {"noarch", self.arch}:
self.logger.debug(f"Skipping spec file: {specFile}")
continue

Expand All @@ -48,10 +48,10 @@ def _readSpecs(self, specFilesPath):
for specPkg in specObj.listPackages:
self.mapPackageToSpec[specPkg] = name

if name in self.mapSpecObjects:
self.mapSpecObjects[name].append(specObj)
else:
if name not in self.mapSpecObjects:
self.mapSpecObjects[name] = [specObj]
else:
self.mapSpecObjects[name].append(specObj)

self.mapSpecFileNameToSpecObj[os.path.basename(specFile)] = specObj

Expand All @@ -73,12 +73,12 @@ def _getListSpecFiles(self, path):
return listSpecFiles

def _getProperVersion(self, depPkg):
if depPkg.compare == "":
if not depPkg.compare:
return self.getHighestVersion(depPkg.package)
specObjs = self.getSpecObjects(depPkg.package)
try:
for obj in specObjs:
verrel = obj.version + "-" + obj.release
verrel = obj.version
if depPkg.compare == ">=":
if LooseVersion(verrel) >= LooseVersion(depPkg.version):
return obj.version
Expand All @@ -88,9 +88,9 @@ def _getProperVersion(self, depPkg):
elif depPkg.compare == "=":
if LooseVersion(verrel) == LooseVersion(depPkg.version):
return obj.version
if LooseVersion(obj.version) == LooseVersion(
depPkg.version
):
x = obj.version.rsplit("-", 1)[0]
y = depPkg.version.rsplit("-", 1)[0]
if LooseVersion(x) == LooseVersion(y):
return obj.version
elif depPkg.compare == "<":
if LooseVersion(verrel) < LooseVersion(depPkg.version):
Expand All @@ -108,19 +108,12 @@ def _getProperVersion(self, depPkg):
raise e

# about to throw exception
availableVersions = ""
availableVersions = []
for obj in specObjs:
availableVersions += (
" " + obj.name + "-" + obj.version + "-" + obj.release
)
raise Exception(
"Could not find package: "
+ depPkg.package
+ depPkg.compare
+ depPkg.version
+ " available specs:"
+ availableVersions
)
availableVersions.append(f"{obj.name}-{obj.version}")
raise Exception("Could not find package: "
f"{depPkg.package}{depPkg.compare}{depPkg.version}"
" available specs: " + " ".join(availableVersions))

def _getSpecObjField(self, package, version, field):
for specObj in self.getSpecObjects(package):
Expand Down Expand Up @@ -306,7 +299,7 @@ def getRPMPackages(self, package, version):

@staticmethod
def compareVersions(p):
return StrictVersion(p.version)
return LooseVersion(p.version)

def getSpecName(self, package):
if package in self.mapPackageToSpec:
Expand All @@ -319,8 +312,7 @@ def getSpecName(self, package):
def isRPMPackage(self, package):
if package in self.mapPackageToSpec:
specName = self.mapPackageToSpec[package]
if specName in self.mapSpecObjects:
return True
return specName in self.mapSpecObjects
return False

def getSecurityHardeningOption(self, package, version):
Expand Down Expand Up @@ -437,7 +429,6 @@ def __init__(self):

def initialize(self):
kver_to_rpm_macros_dict = {
"linux": ["KERNEL_VERSION", "KERNEL_RELEASE", "kernelsubrelease"],
"linux-rt": ["LINUX_RT_KERNEL_VERSION", "LINUX_RT_KERNEL_RELEASE", "linuxrt_kernelsubrelease"],
}

Expand Down
6 changes: 2 additions & 4 deletions support/package-builder/SpecDeps.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -177,13 +177,12 @@ def listRPMfilenames(self, includeDebuginfoRPMs=False):
base_package, version
)
for package in listRPMPackages:
release = SPECS.getData(arch).getRelease(package, version)
buildarch = SPECS.getData(arch).getBuildArch(
package, version
)
filename = os.path.join(
buildarch,
f"{package}-{version}-{release}.{buildarch}.rpm",
f"{package}-{version}.{buildarch}.rpm",
)
output.append(filename)
"""
Expand All @@ -197,8 +196,7 @@ def listRPMfilenames(self, includeDebuginfoRPMs=False):
):
filename = os.path.join(
buildarch,
f"{base_package}-debuginfo-{version}-{release}."
f"{buildarch}.rpm",
f"{base_package}-debuginfo-{version}.{buildarch}.rpm",
)
output.append(filename)

Expand Down
2 changes: 1 addition & 1 deletion support/package-builder/SpecParser.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -675,7 +675,7 @@ def createSpecObject(self):
specObj.specFile = self.specfile
defPkg = self.packages.get("default")
specObj.name = defPkg.name
specObj.version = defPkg.version
specObj.version = f"{defPkg.version}-{defPkg.release}"
specObj.release = defPkg.release
specObj.checksums = defPkg.checksums
specObj.license = defPkg.license
Expand Down
10 changes: 5 additions & 5 deletions support/package-builder/StringUtils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ def getPackageVersionFromURL(self, inputstring):

@staticmethod
def splitPackageNameAndVersion(pkg):
versionindex = pkg.rfind("-")
if versionindex == -1:
raise Exception("Invalid argument")
packageName = pkg[:versionindex]
packageVersion = pkg[versionindex + 1 :] # noqa: E203
packageVersion = pkg.rsplit('-', 1)[0]
if not packageVersion:
raise Exception(f"Invalid argument: {pkg}")
packageName = packageVersion.rsplit("-", 1)[0]
packageVersion = pkg.rsplit(packageName)[1][1:]
return packageName, packageVersion
Loading

0 comments on commit 21bd6f2

Please sign in to comment.