How can I get a list of all repositories and PPAs from the command line into an install script?

I know how to list all packages installed on my system.

But how could I get a list of all repositories and PPA's into a script that I can run on a new machine to replicate the repository setup including the keys?

I know I can look into /etc/apt/sources.list and /etc/apt/sources.list.d, but I'm looking for a way to generate a script that executes all apt-add-repository commands on a new system (that sorts out getting all keys).

Any ideas?

0

16 Answers

You can show everything with:

grep ^ /etc/apt/sources.list /etc/apt/sources.list.d/*

10

Thanks for the pointers. With a little cleanup I got a script that lists the PPAs, but not any other repository:

#! /bin/sh
# listppa Script to get all the PPA installed on a system ready to share for reininstall
for APT in `find /etc/apt/ -name \*.list`; do grep -o "^deb " $APT | while read ENTRY ; do USER=`echo $ENTRY | cut -d/ -f4` PPA=`echo $ENTRY | cut -d/ -f5` echo sudo apt-add-repository ppa:$USER/$PPA done
done

When you call it with listppa > installppa.sh you get a script you can copy on a new machine to reinstall all PPA.

Next stop: do that for the other repositories:

#! /bin/sh
# Script to get all the PPA installed on a system
for APT in `find /etc/apt/ -name \*.list`; do grep -Po "(?<=^deb\s).*?(?=#|$)" $APT | while read ENTRY ; do HOST=`echo $ENTRY | cut -d/ -f3` USER=`echo $ENTRY | cut -d/ -f4` PPA=`echo $ENTRY | cut -d/ -f5` #echo sudo apt-add-repository ppa:$USER/$PPA if [ "ppa.launchpad.net" = "$HOST" ]; then echo sudo apt-add-repository ppa:$USER/$PPA else echo sudo apt-add-repository \'${ENTRY}\' fi done
done

This should do the trick. I needed a question on superuser to figure out the correct regex.

8

I am surprised that the simplest but most effective way to get all enabled binary software sources together with the file they're specified in hasn't been posted yet:

grep -r --include '*.list' '^deb ' /etc/apt/sources.list /etc/apt/sources.list.d/

From all processed files, this will print every line starting with deb. This excludes commented lines as well as deb-src lines to enable source code repositories.

It searches really only all *.list files that will be parsed by apt, but e.g. no *.list.save files used for backup or others with illegal names.

---

If you want a shorter but possibly only in 99.9% of all cases correct output that may search too much files (includes all /etc/apt/sources.list* files and directories, not only /etc/apt/sources.list and `/etc/apt/sources.list.d/*), you could also use this:

grep -r --include '*.list' '^deb ' /etc/apt/sources.list*

Unless there are files that shouldn't be there, the output will be the same.

---

An example output on my machine would be this:

/etc/apt/sources.list:deb wily main restricted
/etc/apt/sources.list:deb wily-updates main restricted
/etc/apt/sources.list:deb wily universe
/etc/apt/sources.list:deb wily-updates universe
/etc/apt/sources.list:deb wily multiverse
/etc/apt/sources.list:deb wily-updates multiverse
/etc/apt/sources.list:deb wily-backports main restricted universe multiverse
/etc/apt/sources.list:deb wily-security main restricted
/etc/apt/sources.list:deb wily-security universe
/etc/apt/sources.list:deb wily-security multiverse
/etc/apt/sources.list:deb wily partner
/etc/apt/sources.list.d/maarten-fonville-ubuntu-ppa-wily.list:deb wily main
/etc/apt/sources.list.d/webupd8team-ubuntu-tor-browser-wily.list:deb wily main
/etc/apt/sources.list.d/fossfreedom-ubuntu-indicator-sysmonitor-wily.list:deb wily main
/etc/apt/sources.list.d/getdeb.list:deb wily-getdeb apps

---

If you want prettier output, let's pipe it through sed:

grep -r --include '*.list' '^deb ' /etc/apt/ | sed -re 's/^\/etc\/apt\/sources\.list((\.d\/)?|(:)?)//' -e 's/(.*\.list):/\[\1\] /' -e 's/deb http:\/\/ppa.launchpad.net\/(.*?)\/ubuntu .*/ppa:\1/'

And we will see this:

deb wily main restricted
deb wily-updates main restricted
deb wily universe
deb wily-updates universe
deb wily multiverse
deb wily-updates multiverse
deb wily-backports main restricted universe multiverse
deb wily-security main restricted
deb wily-security universe
deb wily-security multiverse
deb wily partner
[maarten-fonville-ubuntu-ppa-wily.list] ppa:maarten-fonville/ppa
[webupd8team-ubuntu-tor-browser-wily.list] ppa:webupd8team/tor-browser
[fossfreedom-ubuntu-indicator-sysmonitor-wily.list] ppa:fossfreedom/indicator-sysmonitor
[getdeb.list] deb wily-getdeb apps

4

Run the following command:

apt-cache policy | grep http | awk '{print $2" "$3}' | sort -u

Source

4

Here is my script, "list-apt-repositories", which lists all repositories in "/etc/sources.list" and "/etc/sources.list.d/*.list". You can add --ppa-only to show only the PPAs. PPAs are automatically transformed to ppa:USER/REPO format.

The relevant parts are the 5 lines in list_sources and list_ppa functions, the rest is just boilerplate to wrap it in a handy shell script.

list-apt-repositories:

#!/bin/sh
usage () { cat >&2 <<USAGE
$0 [--ppa-only]
Options: --ppa-only only list PPAs
USAGE exit $1
}
list_sources () { grep -hE '^deb\s' /etc/apt/sources.list /etc/apt/sources.list.d/*.list |\ sed '/ppa/ s/deb //g' |\ sed -re 's#([^/]+)/([^/]+)(.*?)$#ppa:\1/\2#g'
}
list_ppa () { list_sources | grep '^ppa:'
}
generate=list_sources
while test -n "$1"
do case "$1" in -h|--help) usage 1;; --ppa-only) generate=list_ppa;; *) printf -- "Unknown argument '$1'\n" >&2 usage 2 ;; esac shift
done
$generate

And to make an install script, pipe into another script "make-apt-repository-install-script". The generated script supports the -y/--yes argument for non-interactive use (see add-apt-repository(1)).

make-apt-repository-install-script:

#!/bin/sh
if test -n "$1"
then cat >&2 <<USAGE
Usage: $0 < PATH_TO_LIST_OF_REPOS list-apt-repositories [--ppa-only] | $0
No options recognized.
Reads list of repositories from stdin and generates a script to install them
using \`add-apt-repository(1)\`. The script is printed to stdout.
The generated script supports an optional
\`-y\` or \`--yes\` argument which causes the \`add-apt-repository\` commands
to be run with the \`--yes\` flag.
USAGE exit 1
fi
cat <<INSTALL_SCRIPT
#!/bin/sh
y=
case "\$1" in -y|--yes) y=\$1;; '') y=;; *) printf '%s\n' "Unknown option '\$1'" "Usage: \$0 [{-y|--yes}]" >&2 exit 1 ;;
esac
INSTALL_SCRIPT
xargs -d'\n' printf "add-apt-repository \$y '%s'\n"

Again, the important part is the xargs command on the last line, the rest is boilerplate.

1

I use this command to list all configured software sources (repositories), including currently disabled ones:

cat /etc/apt/sources.list; for X in /etc/apt/sources.list.d/*; do echo; echo; echo "** $X:"; echo; cat $X; done

I use this primarily for troubleshooting; this can certainly be incorporated into scripts but you may want to narrow /etc/apt/sources.list.d/* to /etc/apt/sources.list.d/*.list so you only get currently enabled software sources.

1

So, doing some digging, we have AptPkg::Class.

So using perl we can do something simple like this..

perl -MAptPkg::Cache -MData::Dumper -E'say Dumper [AptPkg::Cache->new->files()]' | less

This gets us a list of all the AptPkg::Class::PkgFile packages. You could probably generate the apt-add-repository commands with that.

Here is a one liner:

find /etc/apt/sources.list* -type f -iname "*.list" -exec grep -viE '(^#|^$)' {} \; -print | column -tx
deb bionic main restricted
deb bionic-updates main restricted
deb bionic universe
deb bionic-updates universe
deb bionic multiverse
deb bionic-updates multiverse
deb bionic-backports main restricted universe multiverse
deb bionic-security main restricted
deb bionic-security universe
deb bionic-security multiverse
/etc/apt/sources.list
deb bionic nginx
deb-src bionic nginx
/etc/apt/sources.list.d/nginx.list

1

Using add-apt-repository from software-properties-common, it is as simple as:

add-apt-repository --list

The output can easily be supplied back to add-apt-repository command to recreate the sources.

However, it only lists deb sources. If you are interested also in ppas, then the other answers on this question will be more useful.

Availability

It appears that the --list option was only available since version 0.99.0+ of software-properties-common, which is available by default starting from Ubuntu 20.10 (Groovy). So you will either need to update your version of the software, or upgrade your distro to atleast 20.10.

4

will give you a list of all PPAs for your version of Ubuntu. Here is a generated list without source files and no samsung printer ppa:

#------------------------------------------------------------------------------#
# OFFICIAL UBUNTU REPOS #
#------------------------------------------------------------------------------#
###### Ubuntu Main Repos
deb yakkety main restricted universe multiverse
###### Ubuntu Update Repos
deb yakkety-security main restricted universe multiverse
deb yakkety-updates main restricted universe multiverse
deb yakkety-proposed main restricted universe multiverse
deb yakkety-backports main restricted universe multiverse
###### Ubuntu Partner Repo
deb yakkety partner
#------------------------------------------------------------------------------#
# UNOFFICIAL UBUNTU REPOS #
#------------------------------------------------------------------------------#
###### 3rd Party Binary Repos
#### Flacon PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F2A61FE5
deb yakkety main
#### Gimp PPA -
## Run this command: sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 614C4B38
deb yakkety main
#### Google Chrome Browser -
## Run this command: wget -q -O- | sudo apt-key add -
deb [arch=amd64] stable main
#### Google Earth -
## Run this command: wget -q -O- | sudo apt-key add -
deb [arch=amd64] stable main
#### Highly Explosive PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93330B78
deb yakkety main
#### JDownloader PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6A68F637
deb yakkety main
#### Lazarus -
## Run this command: gpg --keyserver hkp://pgp.mit.edu:11371 --recv-keys 6A11800F && gpg --export --armor 0F7992B0 | sudo apt-key add -
deb lazarus-stable universe
#### LibreOffice PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1378B444
deb yakkety main
#### MEGA Sync Client -
deb ./
#### MKVToolnix -
## Run this command: wget -q -O- | sudo apt-key add -
deb ./
#### Mozilla Daily Build Team PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 247510BE
deb yakkety main
#### muCommander -
## Run this command: sudo wget -O - | sudo apt-key add -
deb stable main non-free contrib
#### Opera -
## Run this command: sudo wget -O - | sudo apt-key add -
deb stable non-free
#### Oracle Java (JDK) Installer PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886
deb yakkety main
#### PlayDeb -
## Run this command: wget -O- | sudo apt-key add -
deb yakkety-getdeb games
#### SABnzbd PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4BB9F05F
deb yakkety main
#### SimpleScreenRecorder PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 283EC8CD
deb yakkety main
#### Steam for Linux -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F24AEA9FB05498B7
deb [arch=i386] precise steam
#### Syncthing -
## Run this command: curl -s | sudo apt-key add -
deb syncthing release
#### Tor: anonymity online -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 886DDD89
deb yakkety main
#### Unsettings PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 0FEB6DD9
deb yakkety main
#### VirtualBox -
## Run this command: wget -q -O- | sudo apt-key add -
deb yakkety contrib
#### Webmin -
## Run this command: wget -O- | sudo apt-key add -
deb sarge contrib
#### WebUpd8 PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4C9D234C
deb yakkety main
#### Xorg Edgers PPA -
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8844C542
deb yakkety main
here is a generated list without source files and no samsung printer ppa
#### Yuuguu -
deb hardy multiverse

To have it add ppa.launchpad.net lines as ppa:$USER/$PPA. Add other repositories with their full line from *.list files. No dupe lines.

#!/bin/bash
# My ~/bin/mk_repositories_restore_script
mkdir -p ~/bin
x=~/bin/restore_repositories
echo \#\!/bin/bash > $x
chmod u+x $x
( for APT in $( find /etc/apt/ -name \*.list ) do sed -n -e '/^deb /{ /ppa\.launchpad/s/\(.*\/\/[^\/]*.\)\([^ \t]*\)\(.*$\)/sudo apt-add-repository ppa:\2/p; /ppa\.launchpad/!s/\(deb[ \t]*\)\(.*$\)/sudo apt-add-repository \2/p; }' $APT done
) | sort | uniq | tee -a ~/bin/restore_repositories

Ripgrep version for completeness:

rg --iglob '*.list' '^deb(-src)? ' /etc/apt/sources.list /etc/apt/sources.list.d/

Thanks BobDodds!
If anybody would be interested, I have updated your code a little (hope you don't mind)..
This script will type out only user added PPAs (/etc/apt/sources.list.d).

 #!/bin/bash # My ~/bin/mk_repositories_restore_script mkdir -p ~/bin x=~/bin/restore_repositories echo \#\!/bin/bash > $x chmod u+x $x ( for APT in $( find /etc/apt/ -name \*.list ) do sed -n -e '/^deb /{ /ppa\.launchpad/s/\(.*\/\/[^\/]*.\)\([^ \t]*\)\(.*\/ubuntu.*$\)/ppa:\2/p; }' $APT done ) | sort | uniq | tee -a ~/bin/restore_repositories

sed -r -e '/^deb /!d' -e 's/^([^#]*).*/\1/' -e 's/deb http:\/\/ppa.launchpad.net\/(.+)\/ubuntu .*/ppa:\1/' -e "s/.*/sudo add-apt-repository '&'/" /etc/apt/sources.list /etc/apt/sources.list.d/*

That does not generate commands to enable possible source repositories (deb-src) though.

In order to list the repositories, I will provide a short method similar to some already posted, also filtering out comments using a regex: ^[^#] ("at the line start, no commented lines"):

grep "^[^#]" /etc/apt/sources.list /etc/apt/sources.list.d/*

Install ppa-purge

apt install ppa-purge

Then get ppa list by tab completion...

ppa-purge -o (hit Tab key twice)

1

Your Answer

Sign up or log in

Sign up using Google Sign up using Facebook Sign up using Email and Password Submit

Post as a guest

Post Your Answer Discard

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy