Packaging: Difference between revisions

From iPhone Development Wiki
(more detail about scripts)
m (Dragon does this automatically now (doesn't theos, too?))
 
(16 intermediate revisions by 5 users not shown)
Line 2: Line 2:


Cydia is based on Debian APT ([https://en.wikipedia.org/wiki/Advanced_Packaging_Tool Advanced Packaging Tool]), so a lot of general documentation about packaging for APT also applies to packaging for Cydia.
Cydia is based on Debian APT ([https://en.wikipedia.org/wiki/Advanced_Packaging_Tool Advanced Packaging Tool]), so a lot of general documentation about packaging for APT also applies to packaging for Cydia.
''To do: bring over http://iphonedevwiki.net/index.php/MobileSubstrate_Pitfalls#Packaging too.''
== Best Practices ==
* Do not create mobile-owned files and/or directories in your package. Stay out of mobile's home directory!
** All package files are installed as root. Your software should create any required files or directories at runtime.
** This is doubly important for preferences. A user's preferences do not belong in a package: if preferences are stored in the package, they will be overwritten when you release an update, and deleted when the user uninstalls your software (even temporarily!)
* Do not use postinst/preinst/extrainst_ for file management purposes!
** Do not store in the package files or directories that your software could create.
** Do not enforce permissions that your package should contain. dpkg uses an expressive packaging format that has support for permissions, ownership, and links. Use that support!
* Make sure your package doesn't include hidden .DS_Store, ._*, or thumbs.db files.
** <code>export COPYFILE_DISABLE=1</code> will disable resource forks (._* files) being included on OS X with Theos.
** Adding <code>find . -name '.DS_Store' -type f -delete</code> to your package staging commands in Theos will ensure they are deleted before packaging.


== Tools ==
== Tools ==
''To do: add existing tools, on which platforms they are available, how to use them.''
 
=== dpkg-deb ===
 
Likely the most ideal software to use for packaging .debs. Part of the dpkg suite.
 
Some repo software requires the -Zgzip flag, so I would advise using this.
 
Example usage:
 
`dpkg-deb --build -Zgzip -z9 my/tweak/staging/directory packages/`
 
 
This will build the contents of `my/tweak/staging/directory` (see #Staging) into a package located in the folder `packages`.
 
" To do: Add other utilities (.pl, .py, et al.) usage, etc). "


== DEBIAN folder ==
== DEBIAN folder ==
Line 10: Line 38:
=== Control file ===
=== Control file ===


''To do: add layout of fields, what do they mean, how they are used. Refer to [http://www.saurik.com/id/7 saurik's post] and links at the bottom.''
{|
!width="21%"| Field
!width="30%"| Example
!width="47%"| Description
|-
| Package
| <code>com.yourcompany.tweak</code>
| Unique Identifier for your tweak. Should typically follow the standard format to avoid conflicting names
|-
| Name
| <code>MyCoolTweak</code>
| Name of your tweak, as displayed in package managers
|-
| Depends
| <code>firmware (&gt;= 13.0), mobilesubstrate, preferenceloader</code>
| Package identifiers of packages your tweak depends on
|-
| Version
| <code>1.0.0</code>
| Version of your tweak. See Semantic Versioning
|-
| Architecture*
| <code>iphoneos-arm</code>
| Architecture of your package. It’s typically always going to be iphoneos-arm.
|-
| Description
| <code>An awesome MobileSubstrate tweak!</code>
| A short description of your package
|-
| Maintainer
| <code>Username &lt;[email protected]&gt;</code>
| Displayed maintainer name and contact email
|-
| Author
| <code>Username &lt;[email protected]&gt;</code>
| Displayed author name and contact email
|-
| Section
| <code>Tweaks</code>
| Section (as displayed in the category browser per repo) the tweak falls under
|-
| Installed-Size*
| <code>5392</code>
| Unpacked size of your package. Your packager (Theos, et al.) should generate this.
|-
| Tag*
| <code>cydia::commercial, compatible::ios13</code>
| Tags for Cydia (version compatibility, should we color it blue, etc)
|-
| Icon
| <code>https://mycoolwebsite.com/com.yourcompany.tweak/icon@3x.png</code>
| Link (url) to an image to be used to represent your package
|-
| Depiction
| <code>https://mycoolwebsite.com/com.yourcompany.tweak/</code>
| Link to a webpage displayed in your package manager
|}
 
Fields with a * are typically added by your repo. Verify yourself with your repo as to which fields need to be added by you and which are added by the repository.


If you need to list dependencies or conflicting packages, Debian's packaging manual may be useful (because Cydia packaging is based on Debian packaging): [https://www.debian.org/doc/debian-policy/ch-relationships.html#s-depsyntax Syntax of relationship fields], [https://www.debian.org/doc/debian-policy/ch-relationships.html#s-binarydeps Dependencies], [https://www.debian.org/doc/debian-policy/ch-relationships.html#s-conflicts Conflicts] -- or if you're submitting this package to a default repository, you can just ask your repository maintainer for help with this.
If you need to list dependencies or conflicting packages, Debian's packaging manual may be useful (because Cydia packaging is based on Debian packaging): [https://www.debian.org/doc/debian-policy/ch-relationships.html#s-depsyntax Syntax of relationship fields], [https://www.debian.org/doc/debian-policy/ch-relationships.html#s-binarydeps Dependencies], [https://www.debian.org/doc/debian-policy/ch-relationships.html#s-conflicts Conflicts] -- or if you're submitting this package to a default repository, you can just ask your repository maintainer for help with this.
For information on the package version format and how versions are sorted, see [http://man7.org/linux/man-pages/man5/deb-version.5.html this page] of the Debian packaging manual.


=== Other files ===
=== Other files ===


You can add <code>preinst</code>, <code>postinst</code>, <code>extrainst_</code>, <code>prerm</code>, and <code>postrm</code> files to run scripts at various points of the package installation and uninstallation lifecycle. It's important to be conservative with these scripts - as [[Best Practices]] explains, "Do not use postinst/preinst/extrainst_ for file management purposes! Do not store in the package files or directories that your software could create. Do not enforce permissions that your package should contain. dpkg uses an expressive packaging format that has support for permissions, ownership, and links. Use that support!"
You can add <code>preinst</code>, <code>postinst</code>, <code>extrainst_</code>, <code>prerm</code>, and <code>postrm</code> files to run scripts at various points of the package installation and uninstallation lifecycle. It's important to be conservative with this code - as [[Best Practices]] explains, "Do not use postinst/preinst/extrainst_ for file management purposes! Do not store in the package files or directories that your software could create. Do not enforce permissions that your package should contain. dpkg uses an expressive packaging format that has support for permissions, ownership, and links. Use that support!"
 
If your package scripts have problems, especially upon uninstallation, this can be very frustrating for your user because that can be very hard to fix for them. You don't want to cause the dreaded [http://theiphonewiki.com/wiki/Cydia_Errors#subprocess_pre-removal_script_returned_error_exit_status_.5Bnumber.5D "subprocess pre-removal script returned error" and "Sub-process /usr/bin/dpkg returned an error code"] error messages. A related unfortunate reality is that if your package has complex scripts, that can increase the chances that if a pirate repository poorly repackages your tweak, somebody doing "try before they buy" can run into those error messages and get frustrated.


Documentation about these scripts in the Debian packaging manual: [https://www.debian.org/doc/manuals/debian-faq/ch-pkg_basics.en.html#s-maintscripts basics] and [https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html more info].
Documentation about these scripts in the Debian packaging manual: [https://www.debian.org/doc/manuals/debian-faq/ch-pkg_basics.en.html#s-maintscripts basics] and [https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html more info].


''To do: Consider [https://ghostbin.com/paste/jukek this paste] as one example. Explain how it fits into Debian packaging (how it interacts with apt/aptitude/dpkg/Cydia).''
==== extrainst_ ====
 
<pre style="white-space: pre-wrap;">
I get a lot of requests for things like "I want to run stuff after the install, but only on install, not on upgrade". APT/dpkg is miserable at this. Its sad, but true. It has a file called postinst, but postinst is really designed for reconfiguring a package, not for installing it: its called /long after/ the package is installed (like, APT will go on and install more things and then queue up the postinst files to the very end) and it doesn't /really/ know anything about the package.
 
Example: in a postinst you know that the previous configuration on disk was from version 1.0, but not that 1.0 itself was actually on disk. And if you are a new install it still tells you that /you're/ already the configuration there (which is really just a stupid semantic).
 
/So/, after dealing with an issue in a package with _BigBoss_, I have decided to do something about this: I have added to dpkg a new script called extrainst_. This script takes one of two arguments: install or upgrade (in which case you also get the version you are upgrading from), and is designed to completely ignore configuration files (so if you were previously in a configuration file only state that isn't considered "installed"). I believe that this new script now provides a very simple conceptualization for how you do "extra stuff" during an install.
 
In fact, if you are previously coming from Installer, this script will likely feel like "ahhhh".
 
To use this new script (which yes, really ends in an underscore) your package should Pre-Depends: dpkg (>= 1.13.25-5). This will make it so that that version of dpkg is actually available during the install and is used to run the scripts. (Yes, this will work even during the install using the old version of dpkg to upgrade dpkg and then switching to the new version of dpkg to install your script ;P).
 
If this script fails then the install is considered to not have worked and the installation procedure will get rolled back (files will get removed, replaced back to the backup copies if it was an upgrade, and a million random scripts will get called with irritatingly complicated to remember arguments).
 
(Note that this version of dpkg isn't up right now, but will be up within an hour or two: i.e., long before any of you is likely to bother to try to use it ;P).
 
An example extrainst_:
 
#!/bin/bash
 
if [[ $1 == install || $1 == upgrade ]]; then
    do_something awesome
fi
 
if [[ $1 == upgrade && $2 == 1.0-4 ]]; then
  fix_bug_in old version
fi
 
-J
</pre>
To see the whole discussion about this, see [http://www.telesphoreo.org/pipermail/cydia-packagers/2008-September/000249.html this thread].
 
==== Telling Cydia to respring/reboot etc. after install ====
 
''From [https://ghostbin.com/paste/jukek this paste]''
 
<source lang=bash>
#!/bin/bash
# extrainst_
# postrm
 
## Acceptable parameters for finish
# return - normal behaviour (return to cydia)
# reopen - exit cydia
# restart - reload springboard
# reload - reload springboard
# reboot - reboot device
##
# Historically, 'restart' restarted springboard but did not reload the launchdaemon.
# (reload was used for this purpose.) Now, restart and reload are equivalent.
function finish() {
f="${1}"
 
# No control fd: bail out
[[ -z "${f}" || -z "${CYDIA}" ]] && return
cydia=(${CYDIA})
 
# Cydia control fd version != 1: bail out
[[ ${cydia[1]} -eq 1 ]] || return
 
echo "finish:${f}" >&${cydia[0]}
}
 
if [[ `basename $0` == "extrainst_" ]]; then
case "$1" in
install )
echo Removing
finish restart
;;
upgrade )
echo Removing
finish return
;;
esac
elif [[ `basename $0` == "postrm" ]]; then
# If this script is postrm, you need to do something a bit more complex:
# postrm is called with remove, purge, upgrade, failed-upgrade, abort-install and abort-upgrade
# You probably only want to act on remove and (as this is the actual removal phase.)
case "$1" in
remove )
echo Removing
finish restart
;;
purge )
echo Purging
finish return
;;
abort-install )
echo Aborting install
finish return
;;
upgrade )
echo Upgrading
finish restart
;;
abort-upgrade )
echo Aborting upgrade
finish return
;;
failed-upgrade )
echo Failed upgrading
finish return
;;
esac
fi
 
exit 0
</source>
 
''To do: Explain how it fits into Debian packaging (how it interacts with apt/aptitude/dpkg/Cydia).''
 
''This should be looked at deeper. This is an implementation of communicating via file descriptors to Cydia from a C program <code>int cydiaFd = (int)strtoul(getenv("CYDIA"), NULL, 10); write(cydiaFd, "finish:reboot", 13);</code>''


== Troubleshooting ==
== Troubleshooting ==


=== dpkg-deb ===
=== dpkg-deb ===
=== Infinite upgrade ===
If your Package Manager (Cydia, et al.) continuously shows a pending upgrade, there may be an error in your control file. This can be caused by a missing "Installed-Size" field. If you've packaged your tweak/theme manually, be sure to include this field.

Latest revision as of 03:34, 13 April 2021

Packaging is the step prior to sharing or releasing a tweak/application/tool/library/theme. Here you put all your project files into a single nicely wrapped file that others can install on their devices.

Cydia is based on Debian APT (Advanced Packaging Tool), so a lot of general documentation about packaging for APT also applies to packaging for Cydia.

To do: bring over http://iphonedevwiki.net/index.php/MobileSubstrate_Pitfalls#Packaging too.

Best Practices

  • Do not create mobile-owned files and/or directories in your package. Stay out of mobile's home directory!
    • All package files are installed as root. Your software should create any required files or directories at runtime.
    • This is doubly important for preferences. A user's preferences do not belong in a package: if preferences are stored in the package, they will be overwritten when you release an update, and deleted when the user uninstalls your software (even temporarily!)
  • Do not use postinst/preinst/extrainst_ for file management purposes!
    • Do not store in the package files or directories that your software could create.
    • Do not enforce permissions that your package should contain. dpkg uses an expressive packaging format that has support for permissions, ownership, and links. Use that support!
  • Make sure your package doesn't include hidden .DS_Store, ._*, or thumbs.db files.
    • export COPYFILE_DISABLE=1 will disable resource forks (._* files) being included on OS X with Theos.
    • Adding find . -name '.DS_Store' -type f -delete to your package staging commands in Theos will ensure they are deleted before packaging.

Tools

dpkg-deb

Likely the most ideal software to use for packaging .debs. Part of the dpkg suite.

Some repo software requires the -Zgzip flag, so I would advise using this.

Example usage:

`dpkg-deb --build -Zgzip -z9 my/tweak/staging/directory packages/`


This will build the contents of `my/tweak/staging/directory` (see #Staging) into a package located in the folder `packages`.

" To do: Add other utilities (.pl, .py, et al.) usage, etc). "

DEBIAN folder

Control file

Field Example Description
Package com.yourcompany.tweak Unique Identifier for your tweak. Should typically follow the standard format to avoid conflicting names
Name MyCoolTweak Name of your tweak, as displayed in package managers
Depends firmware (>= 13.0), mobilesubstrate, preferenceloader Package identifiers of packages your tweak depends on
Version 1.0.0 Version of your tweak. See Semantic Versioning
Architecture* iphoneos-arm Architecture of your package. It’s typically always going to be iphoneos-arm.
Description An awesome MobileSubstrate tweak! A short description of your package
Maintainer Username <[email protected]> Displayed maintainer name and contact email
Author Username <[email protected]> Displayed author name and contact email
Section Tweaks Section (as displayed in the category browser per repo) the tweak falls under
Installed-Size* 5392 Unpacked size of your package. Your packager (Theos, et al.) should generate this.
Tag* cydia::commercial, compatible::ios13 Tags for Cydia (version compatibility, should we color it blue, etc)
Icon https://mycoolwebsite.com/com.yourcompany.tweak/[email protected] Link (url) to an image to be used to represent your package
Depiction https://mycoolwebsite.com/com.yourcompany.tweak/ Link to a webpage displayed in your package manager

Fields with a * are typically added by your repo. Verify yourself with your repo as to which fields need to be added by you and which are added by the repository.

If you need to list dependencies or conflicting packages, Debian's packaging manual may be useful (because Cydia packaging is based on Debian packaging): Syntax of relationship fields, Dependencies, Conflicts -- or if you're submitting this package to a default repository, you can just ask your repository maintainer for help with this.

For information on the package version format and how versions are sorted, see this page of the Debian packaging manual.

Other files

You can add preinst, postinst, extrainst_, prerm, and postrm files to run scripts at various points of the package installation and uninstallation lifecycle. It's important to be conservative with this code - as Best Practices explains, "Do not use postinst/preinst/extrainst_ for file management purposes! Do not store in the package files or directories that your software could create. Do not enforce permissions that your package should contain. dpkg uses an expressive packaging format that has support for permissions, ownership, and links. Use that support!"

If your package scripts have problems, especially upon uninstallation, this can be very frustrating for your user because that can be very hard to fix for them. You don't want to cause the dreaded "subprocess pre-removal script returned error" and "Sub-process /usr/bin/dpkg returned an error code" error messages. A related unfortunate reality is that if your package has complex scripts, that can increase the chances that if a pirate repository poorly repackages your tweak, somebody doing "try before they buy" can run into those error messages and get frustrated.

Documentation about these scripts in the Debian packaging manual: basics and more info.

extrainst_

I get a lot of requests for things like "I want to run stuff after the install, but only on install, not on upgrade". APT/dpkg is miserable at this. Its sad, but true. It has a file called postinst, but postinst is really designed for reconfiguring a package, not for installing it: its called /long after/ the package is installed (like, APT will go on and install more things and then queue up the postinst files to the very end) and it doesn't /really/ know anything about the package.

Example: in a postinst you know that the previous configuration on disk was from version 1.0, but not that 1.0 itself was actually on disk. And if you are a new install it still tells you that /you're/ already the configuration there (which is really just a stupid semantic).

/So/, after dealing with an issue in a package with _BigBoss_, I have decided to do something about this: I have added to dpkg a new script called extrainst_. This script takes one of two arguments: install or upgrade (in which case you also get the version you are upgrading from), and is designed to completely ignore configuration files (so if you were previously in a configuration file only state that isn't considered "installed"). I believe that this new script now provides a very simple conceptualization for how you do "extra stuff" during an install.

In fact, if you are previously coming from Installer, this script will likely feel like "ahhhh".

To use this new script (which yes, really ends in an underscore) your package should Pre-Depends: dpkg (>= 1.13.25-5). This will make it so that that version of dpkg is actually available during the install and is used to run the scripts. (Yes, this will work even during the install using the old version of dpkg to upgrade dpkg and then switching to the new version of dpkg to install your script ;P).

If this script fails then the install is considered to not have worked and the installation procedure will get rolled back (files will get removed, replaced back to the backup copies if it was an upgrade, and a million random scripts will get called with irritatingly complicated to remember arguments).

(Note that this version of dpkg isn't up right now, but will be up within an hour or two: i.e., long before any of you is likely to bother to try to use it ;P).

An example extrainst_:

#!/bin/bash

if [[ $1 == install || $1 == upgrade ]]; then
    do_something awesome
fi

if [[ $1 == upgrade && $2 == 1.0-4 ]]; then
   fix_bug_in old version
fi

-J

To see the whole discussion about this, see this thread.

Telling Cydia to respring/reboot etc. after install

From this paste

#!/bin/bash
# extrainst_
# postrm

## Acceptable parameters for finish
# return - normal behaviour (return to cydia)
# reopen - exit cydia
# restart - reload springboard
# reload - reload springboard
# reboot - reboot device
##
# Historically, 'restart' restarted springboard but did not reload the launchdaemon.
# (reload was used for this purpose.) Now, restart and reload are equivalent.
function finish() {
	f="${1}"

	# No control fd: bail out
	[[ -z "${f}" || -z "${CYDIA}" ]] && return
	cydia=(${CYDIA})

	# Cydia control fd version != 1: bail out
	[[ ${cydia[1]} -eq 1 ]] || return

	echo "finish:${f}" >&${cydia[0]}
}

if [[ `basename $0` == "extrainst_" ]]; then
	case "$1" in
	install )
		echo Removing
		finish restart
		;;
	upgrade )
		echo Removing
		finish return
		;;
	esac
elif [[ `basename $0` == "postrm" ]]; then
	# If this script is postrm, you need to do something a bit more complex:
	# postrm is called with remove, purge, upgrade, failed-upgrade, abort-install and abort-upgrade
	# You probably only want to act on remove and (as this is the actual removal phase.)
	case "$1" in
	remove )
		echo Removing
		finish restart
		;;
	purge )
		echo Purging
		finish return
		;;
	abort-install )
		echo Aborting install
		finish return
		;;
	upgrade )
		echo Upgrading
		finish restart
		;;
	abort-upgrade )
		echo Aborting upgrade
		finish return
		;;
	failed-upgrade )
		echo Failed upgrading
		finish return
		;;
	esac
fi

exit 0

To do: Explain how it fits into Debian packaging (how it interacts with apt/aptitude/dpkg/Cydia).

This should be looked at deeper. This is an implementation of communicating via file descriptors to Cydia from a C program int cydiaFd = (int)strtoul(getenv("CYDIA"), NULL, 10); write(cydiaFd, "finish:reboot", 13);

Troubleshooting

dpkg-deb

Infinite upgrade

If your Package Manager (Cydia, et al.) continuously shows a pending upgrade, there may be an error in your control file. This can be caused by a missing "Installed-Size" field. If you've packaged your tweak/theme manually, be sure to include this field.