Skip to content

Commit

Permalink
formalize the test repositories. but test repo into the repo files, b…
Browse files Browse the repository at this point in the history 
…ut disabled. consolidate makefile rules for repos and refactor a bit. added section to dev notes about repo management.
  • Loading branch information
Matthew Wall committed Jul 7, 2024
1 parent 4aeb41b commit 1051d05
Show file tree
Hide file tree
Showing 8 changed files with 95 additions and 27 deletions.
31 changes: 29 additions & 2 deletions DEV_NOTES.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,34 @@ Steps:
make upload-pypi


howto: deb repository ---------------------------------------------------------
howto: update package repositories --------------------------------------------

There are two repositories for each platform, one for production and one for
testing. For example, for debian we have apt and apt-test. When preparing
a debian, redhat, or suse package for release, first do testing by installing
from a local deb/rpm. When you are ready to release to a repo, the steps are:

1) make pull-xxx-repo - this will copy the production repo to local disk

2) make update-xxx-repo - this will copy staged deb/rpm to local repo, then
update repo info and sign with gpg keys

3) make push-xxx-repo - this will copy the local repo to remote test repo

4) make release-xxx-repo - replace production repo with test repo

If you are doing alpha or beta releases, you can push those to the remote test
repo, but do *not* release them. After doing alpha and/or beta releases, be
sure to pull the production repo before you get out of alph/beta, then push to
the test repo. Otherwise you risk pushing alpha/beta releases into the
production repo, which will break many systems.

We must keep alpha/beta releases out of the production repo, since they are
not handled properly by some package management software. For example, yum
thinks that 5.1.0-b6-1 is newer than 5.1.0.


howto: deb repository details -------------------------------------------------

aptly has two different mechanisms for doing a 'publish': switch or update.
we use snapshots, and publish using 'publish switch', rather than publishing
Expand Down Expand Up @@ -291,7 +318,7 @@ for clients to use an apt repo at weewx.com:
echo "deb [arch=all] http://weewx.com/apt/ buster main" | sudo tee /etc/apt/sources.list.d/python3-weewx.list


howto: yum repository ---------------------------------------------------------
howto: yum repository details -------------------------------------------------

create yum repo:
mkdir -p ~/.yum/weewx/{el7,el8,el9}/RPMS
Expand Down
55 changes: 30 additions & 25 deletions makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,14 @@ help: info
@echo " info display values of variables we care about"
@echo " version get version from pyproject.toml and insert elsewhere"
@echo ""
@echo " test run all unit tests"
@echo " SUITE=path/to/foo.py to run only foo tests"
@echo " test-clean remove test databases"
@echo ""
@echo " build-docs build the docs using mkdocs"
@echo " upload-docs upload docs to $(WEEWX_COM)"
@echo " check-docs run weblint on the docs"
@echo ""
@echo " debian-changelog prepend stub changelog entry for debian"
@echo " redhat-changelog prepend stub changelog entry for redhat"
@echo " suse-changelog prepend stub changelog entry for suse"
Expand All @@ -58,37 +66,32 @@ help: info
@echo " check-debian check the debian package"
@echo " check-redhat check the redhat package"
@echo " check-suse check the suse package"
@echo " check-docs run weblint on the docs"
@echo ""
@echo " upload-src upload the src package to $(WEEWX_COM)"
@echo " upload-pypi upload wheel and src package to pypi.org"
@echo " upload-debian upload the debian deb package"
@echo " upload-redhat upload the redhat rpm packages"
@echo " upload-suse upload the suse rpm packages"
@echo ""
@echo " build-docs build the docs using mkdocs"
@echo " upload-docs upload docs to $(WEEWX_COM)"
@echo ""
@echo " release promote staged files on the download server"
@echo ""
@echo " test run all unit tests"
@echo " SUITE=path/to/foo.py to run only foo tests"
@echo " test-clean remove test databases"
@echo ""
@echo " apt repository management"
@echo " pull-apt-repo"
@echo " update-apt-repo"
@echo " push-apt-repo"
@echo " release-apt-repo"
@echo ""
@echo " yum repository management"
@echo " pull-yum-repo"
@echo " update-yum-repo"
@echo " push-yum-repo"
@echo " release-yum-repo"
@echo ""
@echo " suse repository management"
@echo " pull-suse-repo"
@echo " update-suse-repo"
@echo " push-suse-repo"
@echo " pull-suse-repo"
@echo " update-suse-repo"
@echo " push-suse-repo"
@echo " release-suse-repo"
@echo ""

info:
Expand Down Expand Up @@ -424,7 +427,7 @@ apt-repo:
# make local copy of the published apt repository
pull-apt-repo:
mkdir -p ~/.aptly
rsync -Oarvz $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/aptly/ ~/.aptly
rsync -Oarvz --delete $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/aptly/ ~/.aptly

# add the latest version to the local apt repo using aptly
update-apt-repo:
Expand All @@ -438,11 +441,11 @@ update-apt-repo:
push-apt-repo:
find ~/.aptly -type f -exec chmod 664 {} \;
find ~/.aptly -type d -exec chmod 2775 {} \;
rsync -Ortlvz ~/.aptly/ $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/aptly-test
rsync -Ortlvz --delete ~/.aptly/ $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/aptly-test

# copy the testing repository onto the production repository
release-apt-repo:
ssh $(USER)@$(WEEWX_COM) "rsync -Ologrvz /var/www/html/aptly-test/ /var/www/html/aptly"
ssh $(USER)@$(WEEWX_COM) "rsync -Ologrvz --delete /var/www/html/aptly-test/ /var/www/html/aptly"

# 'yum-repo' is only used when creating a new yum repository from scratch
# the index.html is not part of an official rpm repository. it is included
Expand All @@ -458,7 +461,7 @@ yum-repo:

pull-yum-repo:
mkdir -p $(YUM_REPO)
rsync -Oarvz $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/yum/ ~/.yum
rsync -Oarvz --delete $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/yum/ ~/.yum

update-yum-repo:
mkdir -p $(YUM_REPO)/el8/RPMS
Expand All @@ -468,20 +471,21 @@ update-yum-repo:
cp -p $(DSTDIR)/weewx-$(RPMVER).el9.$(RPMARCH).rpm $(YUM_REPO)/el9/RPMS
createrepo $(YUM_REPO)/el9
ifeq ("$(SIGN)","1")
gpg -abs -o $(YUM_REPO)/el8/repodata/repomd.xml.asc $(YUM_REPO)/el8/repodata/repomd.xml
gpg --export --armor > $(YUM_REPO)/el8/repodata/repomd.xml.key
gpg -abs -o $(YUM_REPO)/el9/repodata/repomd.xml.asc $(YUM_REPO)/el9/repodata/repomd.xml
gpg --export --armor > $(YUM_REPO)/el9/repodata/repomd.xml.key
for os in el8 el9; do \
gpg --export --armor > $(YUM_REPO)/$$os/repodata/repomd.xml.key; \
gpg -abs -o $(YUM_REPO)/$$os/repodata/repomd.xml.asc.new $(YUM_REPO)/$$os/repodata/repomd.xml; \
mv $(YUM_REPO)/$$os/repodata/repomd.xml.asc.new $(YUM_REPO)/$$os/repodata/repomd.xml.asc; \
done
endif

push-yum-repo:
find ~/.yum -type f -exec chmod 664 {} \;
find ~/.yum -type d -exec chmod 2775 {} \;
rsync -Ortlvz ~/.yum/ $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/yum-test
rsync -Ortlvz --delete ~/.yum/ $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/yum-test

# copy the testing repository onto the production repository
release-yum-repo:
ssh $(USER)@$(WEEWX_COM) "rsync -Ologrvz /var/www/html/yum-test/ /var/www/html/yum"
ssh $(USER)@$(WEEWX_COM) "rsync -Ologrvz --delete /var/www/html/yum-test/ /var/www/html/yum"

# 'suse-repo' is only used when creating a new suse repository from scratch
# the index.html is not part of an official rpm repository. it is included
Expand All @@ -496,25 +500,26 @@ suse-repo:

pull-suse-repo:
mkdir -p $(SUSE_REPO)
rsync -Oarvz $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/suse/ ~/.suse
rsync -Oarvz --delete $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/suse/ ~/.suse

update-suse-repo:
mkdir -p $(SUSE_REPO)/suse15/RPMS
cp -p $(DSTDIR)/weewx-$(RPMVER).suse15.$(RPMARCH).rpm $(SUSE_REPO)/suse15/RPMS
createrepo $(SUSE_REPO)/suse15
ifeq ("$(SIGN)","1")
gpg -abs -o $(SUSE_REPO)/suse15/repodata/repomd.xml.asc $(SUSE_REPO)/suse15/repodata/repomd.xml
gpg --export --armor > $(SUSE_REPO)/suse15/repodata/repomd.xml.key
gpg -abs -o $(SUSE_REPO)/suse15/repodata/repomd.xml.asc $(SUSE_REPO)/suse15/repodata/repomd.xml
mv $(SUSE_REPO)/suse15/repodata/repomd.xml.asc.new $(SUSE_REPO)/suse15/repodata/repomd.xml.asc
endif

push-suse-repo:
find ~/.suse -type f -exec chmod 664 {} \;
find ~/.suse -type d -exec chmod 2775 {} \;
rsync -Ortlvz ~/.suse/ $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/suse-test
rsync -Ortlvz --delete ~/.suse/ $(USER)@$(WEEWX_COM):$(WEEWX_HTMLDIR)/suse-test

# copy the testing repository onto the production repository
release-suse-repo:
ssh $(USER)@$(WEEWX_COM) "rsync -Ologrvz /var/www/html/suse-test/ /var/www/html/suse"
ssh $(USER)@$(WEEWX_COM) "rsync -Ologrvz --delete /var/www/html/suse-test/ /var/www/html/suse"

# shortcuts to upload everything. assumes that the assets have been staged
# to the local 'dist' directory.
Expand Down
6 changes: 6 additions & 0 deletions pkg/weewx-el.repo
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ name=weewx
baseurl=http://weewx.com/yum/weewx/el$releasever
enabled=1
gpgcheck=1

[weewx-testing]
name=weewx
baseurl=http://weewx.com/yum-test/weewx/el$releasever
enabled=0
gpgcheck=1
6 changes: 6 additions & 0 deletions pkg/weewx-el8.repo
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ name=weewx
baseurl=http://weewx.com/yum/weewx/el8
enabled=1
gpgcheck=1

[weewx-testing]
name=weewx
baseurl=http://weewx.com/yum-test/weewx/el8
enabled=0
gpgcheck=1
6 changes: 6 additions & 0 deletions pkg/weewx-el9.repo
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ name=weewx
baseurl=http://weewx.com/yum/weewx/el9
enabled=1
gpgcheck=1

[weewx-testing]
name=weewx
baseurl=http://weewx.com/yum-test/weewx/el9
enabled=0
gpgcheck=1
6 changes: 6 additions & 0 deletions pkg/weewx-suse.repo
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ name=weewx
baseurl=http://weewx.com/suse/weewx/suse$releasever_major
enabled=1
gpgcheck=1

[weewx-testing]
name=weewx
baseurl=http://weewx.com/suse-test/weewx/suse$releasever_major
enabled=0
gpgcheck=1
6 changes: 6 additions & 0 deletions pkg/weewx-suse12.repo
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ name=weewx
baseurl=http://weewx.com/suse/weewx/suse12
enabled=1
gpgcheck=1

[weewx-testing]
name=weewx
baseurl=http://weewx.com/suse-test/weewx/suse12
enabled=0
gpgcheck=1
6 changes: 6 additions & 0 deletions pkg/weewx-suse15.repo
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ name=weewx
baseurl=http://weewx.com/suse/weewx/suse15
enabled=1
gpgcheck=1

[weewx-testing]
name=weewx
baseurl=http://weewx.com/suse-test/weewx/suse15
enabled=0
gpgcheck=1

0 comments on commit 1051d05

Please sign in to comment.