Quantcast
Channel: linuxium.com.au

New features in 'isorespin.sh'

$
0
0

Canonical publishing new releases of Ubuntu means new release code names, version numbers and updated EOL schedules for existing releases. In turn this drives new versions of 'isorespin.sh' which include support for these latest releases together with any idiosyncrasies of the resultant ISOs that need specific coding work arounds. Inevitably new versions are also required for bug fixes and also as opportunities arise for general improvements.

One such opportunity came as a result of users wanting to respin Ubuntu-Studio ISOs which earlier were not supported as this distro used 'lowlatency' kernels rather than the standard 'generic' kernel. Although adding support for Ubuntu-Studio was relatively easy it did require a change to how kernel packages were handled in general.

In turn this prompted a complete rewrite of how 'rolling' kernels were processed and included important efficiency improvements together with support for the installation of meta kernel packages where appropriate rather than specific kernel versions facilitating easier ongoing upgrades post ISO installation.

Finally three new options ('--debug', '--interactive' and '--dist-upgrade') have been added which improve the overall functionality of 'isorespin.sh'. These have resulted in a rework of the 'usage' text:

linuxium@LINUXIUM:~$ isorespin.sh --help
Usage: /usr/local/bin/isorespin.sh [ -h | -v | --check | --rolling-list ]
       /usr/local/bin/isorespin.sh -i <ISO> [ [ -u | -k <kernel> ] | [ --dist-upgrade | --upgrade ] | ...
       /usr/local/bin/isorespin.sh ... [ --rolling-release | --rolling-release-hwe | --rolling-release-hwe-edge | --rolling-proposed | --rolling-proposed-hwe | --rolling-proposed-hwe-edge | ...
       /usr/local/bin/isorespin.sh ... --rolling-testing | --rolling-testing-hwe | --rolling-testing-hwe-edge | --rolling-unstable | --rolling-unstable-hwe | --rolling-unstable-hwe-edge ] | ...
       /usr/local/bin/isorespin.sh ... -b [ GRUB | GRUB-32 | GRUB-64 | rEFInd | Linuxium ] | -g [ "" | "<kernel boot parameter> ... " ] | -s [ <size>MB | <size>GB ] | -w <directory> | ...
       /usr/local/bin/isorespin.sh ... --key "<repo> ... " | -r "<repo> ... " | -p "<pkg> ... " | -l "<pkg.deb> ... " | -e "<pkg> ... " | -d "<pkg> ... " | -f [ "<file> | <directory> ... " ] | ...
       /usr/local/bin/isorespin.sh ... -c "<cmd> ... " | -o [ "<file> | <directory> ... " ] | -t <template configuration file> | --apollo | --atom | --interactive | --debug ]
linuxium@LINUXIUM:~$ 

The first new option is '--debug'. Most of the options don't include their interaction with the ISO when being executed either on the screen or in the log file. For example when adding the package 'ethtool' the log merely includes a line stating that the package has been added:


The '--debug' option redirects the output from executing the commands behind an option to the log file. Using the same example the complete output from installing the package is now included in the log file:


While including the output from successful option execution may be interesting the key benefit of using '--debug' is when a command run as part of an option fails. Having the full output including the actual error messages in the log file is invaluable for debugging respinning issues.

The second new option has been included as sometimes trying to respin an ISO using complex option combinations fails as typically the consequences of running certain commands or their effect on the ISO are not fully known or easily predictable. In these circumstances respinning interactively would be easier and hence the new option '--interactive'. This option simply drops you into a 'root' shell where you can manually enter commands to modify the ISO:


Simply press 'control-D' when finished to return to respinning the ISO. This option uses the 'script' command to record the interactive session so that it can be included in the log file in full if the '--debug' option is used in conjunction or manipulated to just show a summary of the commands entered for inclusion into the default log file. This command summary is also included in the 'README.isorespin' file that is added to the respun ISO. As a result 'isorespin.sh' now depends on the package 'bsdutils' being installed which should already be the case in most situations. Because 'script' makes a typescript of everything displayed on the terminal it also includes temporary progress text together with cursor movement control codes and colour control codes which may become visible depending on how the log file is viewed. This may not be ideal in every circumstance however it is a compromise believed to be worthwhile considering the functionality gained.

Using the same example as earlier but this time installing the 'ethtool' package interactively having first performed an ''apt update':


the 'cat' command displays the log file created using the '--debug' option without distractions:


Because when performing the 'apt update' command various 'source' files are needed to be downloaded which result in progress text temporarily being displayed on the screen, when the command 'more' is used to display the log file it interprets the 'script' text in the log file differently and shows this colour-highlighted progress text:


Using the command 'view' to examine the log file shows all the text and all the control characters so consequently may not be the best way to view the log file:


For this particular example the summary of the '--interactive' commands that is also included in README.isorespin:


is similar to how this '--interactive' option is documented in the default log file without an additional '--debug' option:


The final new option is '--dist-upgrade'. This is similar to the '--upgrade' option and is best described by the 'man' entry for the commands used by the two options:

upgrade
upgrade is used to install the newest versions of all packages currently installed on the system from the sources enumerated in /etc/apt/sources.list. Packages currently installed with new versions available are retrieved and upgraded; under no circumstances are currently installed packages removed, or packages not already installed retrieved and installed. New versions of currently installed packages that cannot be upgraded without changing the install status of another package will be left at their current version. ...

dist-upgrade
dist-upgrade in addition to performing the function of upgrade, also intelligently handles changing dependencies with new versions of packages; apt-get has a "smart" conflict resolution system, and it will attempt to upgrade the most important packages at the expense of less important ones if necessary. The dist-upgrade command may therefore remove some packages. The /etc/apt/sources.list file contains a list of locations from which to retrieve desired package files. ...
To illustrate the difference we can look at respinning an ISO (Ubuntu 18.04.3) first with the '--upgrade' option together with the '--debug' option which shows that the kernel meta packages are held back:


However using the '--dist-upgrade' option with the '--debug' options shows that new kernel packages will be installed as a result of upgrading the kernel meta packages:


This example again highlights the usefulness of the '--debug' option in understanding what happens as part of respinning an ISO.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf.

Canonical have announced a new point release for Ubuntu 18.04 LTS - 18.04.4 (Bionic Beaver)

$
0
0

Canonical have released the fourth point release of Ubuntu 18.04 Long-Term Support (LTS) as Ubuntu 18.04.4.

I’ve respun the desktop ISO using my ‘isorespin.sh‘ script and created ISOs suitable for Intel Atom and Intel Apollo Lake devices:
Atom (-i ubuntu-18.04.4-desktop-amd64.iso –atom)
Apollo (-i ubuntu-18.04.4-desktop-amd64.iso –apollo)
Please donate if you find these ISOs useful.

Using 'LXPanel' as a UI for Crostini

$
0
0

If you are used to a menu-driven user interface in Linux or find the Chrome OS application launcher not quite to your liking for accessing Crostini Linux applications then one option you could try is LXPanel.

The panel generates a menu for installed applications automatically from '*.desktop' files and can itself be incorporated in its own '.desktop' file which if pinned to the Chrome OS shelf can also be used as a means to start the 'penguin' container after booting.

Unfortunately it is not quite perfect as the panel is displayed in the middle of the screen and doesn't respond well to changing its position under geometry in its panel settings. However you can toggle its visibility by clicking the panel's icon on the shelf. Also closing the panel (by right clicking the icon) only closes the 'LXPanel' application in Chrome OS so to terminate it fully you need to use 'killall lxpanel' in a terminal session.


To set it up first ensure 'LXPanel' is installed in your container: assuming you are using the default Debian-based 'penguin' or a similar derivative (e.g. Ubuntu) container simply enter 'sudo apt install lxpanel'.

Once installed open up a terminal session and create a file '~/.local/share/applications/lxpanel.desktop' (or '/usr/share/applications/lxpanel.desktop') containing:
[Desktop Entry]
Version=1.0
Name=LX Panel
Comment=Opens LX Panel and to terminate use killall lxpanel
Exec=lxpanel
Type=Application
Terminal=false
Icon=<icon path here>

For the icon you can use any icon '.png' you prefer. For example if you have already previously installed LXDE then you could use the icon '/usr/share/lxde/images/lxde-icon.png'. If you don't have a suitable LXDE icon then you can always download this one and edit the 'lxpanel.desktop' file accordingly (e.g. 'Icon=/home/linuxiumcomau/247px-LXDE-logo.svg.png').

After a minute or so your new 'LXPanel' application should appear in the Crostini applications:


You can now right-click on its icon and pin it to the shelf:


To start it click on the shelf icon and 'lxpanel' will appear in the centre of the screen:


Its visibility can be toggled by clicking its icon on the shelf. As previously mentioned closing the panel by right-clicking the shelf icon only closes the application in Chrome OS so to terminate it fully you need to enter 'killall lxpanel' in a terminal session.

Please donate if you find this guide useful using the following link http://goo.gl/nXWSGf.

Creating and Installing a Server Respun Ubuntu Desktop ISO

$
0
0

Whilst Canonical release'server' images of Ubuntu getting them to boot or install on Intel mini PCs can be difficult especially if the mini PC is limited by the BIOS enforcing a 32-bit bootloader. As a result there have been many requests for 'isorespin.sh' to respin an Ubuntu server ISO. However the structure of a server ISO is different to a desktop ISO and as a result is incompatible with the script. So I've always suggested as a workaround to first respin and install the desktop ISO and then purge the desktop packages and install the required server packages.

But what if you try and respin a desktop ISO as a server ISO?

Well this is possible however you need to add the package 'casper' in order to boot it. You also need to create a user as part of respinning otherwise you won't be able to login.

The problem is that even after successfully booting a 'pseudo' server ISO installing it is rather tedious.

Therefore I'm developing a text based script to simplify the installation and I've documented the current status below.

First you need to respin the Ubuntu desktop ISO using 'isorespin.sh' with the following options:

isorespin.sh \
-i ubuntu-18.04.4-desktop-amd64.iso \
--atom \
-e ubuntu-desktop \
-p "ubuntu-server casper gdisk network-manager linux-generic-hwe-18.04" \
-f linservin.sh \
-c "useradd -c ubuntu -d /home/ubuntu -m -g users -s /bin/bash ubuntu" \
-c "sed -i '/^root/ a ubuntu ALL=(ALL:ALL) ALL' /etc/sudoers" \
-c "passwd -de ubuntu"

To explain further the options included are:
'-i ubuntu-18.04.4-desktop-amd64.iso' is the desktop ISO I've currently been testing with.
'--atom' for an Intel Atom based device. Alternatively use '--apollo' for those relevant devices that don't boot the standard desktop ISO or '-b rEFInd' to include the 'rEFInd' boot manager.
'-e ubuntu-desktop' to purge (remove) the packages tagged with task 'ubuntu-desktop' which will remove the graphical interface.
'-p ubuntu-server' to install all the packages included in the metapackage 'ubuntu-server'.
'-p casper' to provice the scripts required to boot the ISO.
'-p gdisk' is required by my installation script in order to format the storage.
'-p network-manager' again required by the installation script in order to have internet access.
'-p linux-generic-hwe-18.04' as it was removed during the desktop purge.
(Note: these packages have been combined into one '-p' option)
'-f linservin.sh' includes my installation script (see below).
-c "useradd ..." this command creates the user 'ubuntu' for the respun ISO. Any user can be created.
-c "sed ..." allows the 'ubuntu' user 'sudo' access.
-c "passwd ..." allows a password of choice to be created on initial boot.

A further '--debug' option can be added so that the resultant logfile shows details of all the package dependencies used in the purge and install commands. Full documentation on the options can be found here.

My 'Linuxium server install' script 'linservin.sh' can be downloaded by clicking on this link.

Using the above options creates the 'pseudo' server ISO. I've uploaded a respun ISO here which you can use for testing and I've renamed it as 'linuxium-atom-ubuntu-18.04.4-server-amd64.iso' just to avoid confusion.

Next create a 'LiveUSB' by writing the ISO to a USB using 'dd':
dd if=linuxium-atom-ubuntu-18.04.4-server-amd64.iso of=/dev/sdX bs=4M
where 'X' is the drive letter for the USB.


Then boot from the 'LiveUSB' using 'Try Ubuntu without installing' and you'll have a 'live' server ISO which you can use similar to any live ISO. Also don't try clicking the 'Install Ubuntu' option as this will fail as the required packages have already been removed by the purge command.

Prior to attempting an installation you must have a working internet connection as packages are required to be downloaded. It is not important whether it is by ethernet or wifi and the 'nmcli' command can be used to configure a connection.

Finally to install the server ISO simply run 'linservin.sh' which is included on the ISO in 'usr/local/bin':


The script can also be run with options:
linuxium@LINUXIUM:~$ linservin.sh -h
Usage: /usr/local/bin/linservin.sh [ OPTIONS ]
where OPTIONS include '-h' for 'help'
'-v' for 'version'
'-c' for 'check' (version)
'-y' run automatically with best guessed values (dangerous)
linuxium@LINUXIUM:~$
which are self explanatory.

The script then shows the variables it will use for the installation together with values it determines may be the most appropriate and then allows you to select what you want:


It will then ask for confirmation to start the installation (bearing in mind this will completely overwrite the existing hard drive of the device) and it requires a (case sensitive) 'Y' to continue.

The following video shows the installation on an Intel Compute Stick (STCK1A32WFC):



Importantly the installer will detect whether the bootloader is 32-bit for 64-bit and install GRUB accordingly.

The installation progress is recorded in a logfile 'linservin.log' which is also copied to the installed filesystem and is available under '/var/log/installer'.

If the 'server' ISO proves popular I'm considering adding a '--server' option to 'isorespin.sh' to facilitate the respinning of server ISOs. Otherwise feedback is appreciated to improve the current 'linservin.sh' script.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.

Using 'isorespin.sh' to create a server ISO from an Ubuntu desktop ISO

$
0
0

To simplify the creation of a server ISO by respinning an Ubuntu desktop ISO I've added a new option '--server'. This option is compatible with existing options so you can create a server ISO that works on both 32-bit and 64-bit bootloaders found on various low cost Intel Atom mini PCs or one that works on the more recent Apollo and Gemini Lake mini PCs.

Invocation is as simple as adding '--server' to your 'isorespin.sh' command. For example to create a 'vanilla' server ISO from an Ubuntu 18.04.4 desktop ISO enter:
isorespin.sh -i ubuntu/ubuntu-18.04.4-desktop-amd64.iso --server
Here is a (speeded up) video showing this example in action:



The '--server' option is only supported with Ubuntu 18.04 and 20.04 desktop ISOs at the moment. This is because whilst removing the 'ubuntu-desktop' task essentially creates the base for a server ISO, it still leaves a small number of residual packages that need purging. This new option removes the bulk of these and those that are left are really insignificant.

The option now creates 'ubuntu' as the default user without a password, similar to how the standard desktop ISO works. It also downloads the latest version of 'linservin.sh' to '/usr/local/bin' for convenience when installing.

The latest version of 'isorespin.sh' can be downloaded from here.

You can install the server ISO using the 'linservin.sh' which I've also updated. Now rather than having to enter 'c' to 'change to the detected value' you simply hit enter to accept the default or enter your own value if required:


The script also now creates a swap file which is adjusted in size to be appropriate for the device's disk and memory sizes.

An updated 'installation' video can be seen here:


which again is speeded up in parts to make it more watchable.

The latest version of 'linservin.sh' can be downloaded from here.

Please donate if you find the scripts useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.












linuxminipcs.com

$
0
0

A while back I created the website www.linuxminipcs.com which I hoped would become the 'go to' site for all matters Linix on mini PCs. Unfortunately it didn't go viral! Neither has it got much attention and the reality is that it meets the criteria of being a failed project: it has become a burden both financially and resourcefully as well as technically.

So it is time to close it down and in early May the site will quietly disappear.

Please donate if you find my remaining sites useful using the following link http://goo.gl/nXWSGf as everything helps with support costs.

Canonical have announced the release of Ubuntu 20.04 LTS - 20.04 (Focal Fossa)

$
0
0

Canonical have announced the latest release of Ubuntu 20.04 Long-Term Support (LTS) as Ubuntu 20.04 LTS (Focal Fossa).

I’ve respun the desktop ISO using my ‘isorespin.sh‘ script and created ISOs suitable for Intel Atom and Intel Apollo Lake devices:

Atom (-i ubuntu-20.04-desktop-amd64.iso –atom)
Apollo (-i ubuntu-20.04-desktop-amd64.iso –apollo)

I've also respun the desktop ISO with the '--server' option to create a pseudo server ISO suitable for Intel devices with a 32-bit bootloader:

Server (-i ubuntu-20.04-desktop-amd64.iso –atom)



Also announced are the official 20.04 flavours of Ubuntu including Lubuntu which I've also respun to created an ISO suitable for Intel Atom devices:

Atom (-i lubuntu-20.04-desktop-amd64.iso –atom)

Please donate if you find these ISOs useful.

Comparing packages between ISOs or between an ISO and those locally installed using 'isocomparepkgs.sh'

$
0
0


This is the first of three posts introducing a couple of new 'ISO' tools that I've developed to complement my 'isorespin.sh' script.

The first tool is 'isocomparepkgs.sh' which is a script to compare installed packages between two Ubuntu (or Ubuntu flavour), Linux Mint, neon, elementary, BackBox or Peppermint desktop ISOs or Ubuntu live server ISOs or between one such ISO and the locally installed packages on the system where the script is run.
linuxium@LINUXIUM:~$ isocomparepkgs.sh
Usage: isocomparepkgs.sh <ISO> [<ISO>]
linuxium@LINUXIUM:~$
The usage syntax is straightforward: you either run the script against two ISOs to get a package comparison of those installed in each or you run the script against one ISO to get a comparison of packages in the ISO verses those locally installed.

For example, say you want to see the package difference between the original Ubuntu 18.04 ISO and the latest point release of 18.04.4. Simply enter the command:

isocomparepkgs.sh ubuntu-18.04-desktop-amd64.iso ubuntu-18.04.4-desktop-amd64.iso

The resultant output looks like this:



with the logfile 'isocomparepkgs.log' also including a side-by-side comparison:


It is important to note that the script only shows the difference between installed packages based on their name and not their version or release. Also it only shows packages and not userspace file differences. 

So the difference between a respun Ubuntu 18.04.4 ISO using the 'atom' option and the original ISO only shows the additional bluetooth package and not the wifi and ALSA UCM files which were also added as a result of respinning:


When comparing packages in an ISO with those locally installed it is important to remember than an ISO also includes several packages used only as part of the installation process:


and that the list of locally installed packages also includes all the dependent packages which can make it difficult to determine the actual packages used in the install commands:


which is a why the second post in this series may be of interest.

The initial release of 'isocomparepkgs.sh' can be downloaded from here.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.

How to create an ISO that mimics the installed packages from an ISO or those locally installed using 'isomimicpkgs.sh'

$
0
0
This is the second of three posts introducing a couple of new 'ISO' tools that I've developed to complement my 'isorespin.sh' script.

The second tool is 'isomimicpkgs.sh' which is a script to compare installed packages in two Ubuntu (or Ubuntu flavour), Linux Mint, neon, elementary, BackBox or Peppermint desktop ISOs or Ubuntu live server ISOs, or one such ISO with locally installed packages and identify which packages need to be added to the second ISO to mimic the first ISO or added to the first ISO to mimic those locally installed. In otherwords it will show how to update an ISO with either the packages found in another ISO or with those locally installed.

For example, say you want to create a test enviroment with the same packages as those locally installed. Normally you would install the same Ubuntu release ISO and then install the packages one by one in order to replicate the enviroment. However this is not so simple when you have a lot of packages installed.

You can start by finding the ISO that was used for the initial install by looking at the contents of the file '/var/log/installer/media-info':

linuxium@LINUXIUM:~$ cat /var/log/installer/media-info 
Ubuntu 18.04 LTS "Bionic Beaver" - Release amd64 (20180426)linuxium@LINUXIUM:~$ 
linuxium@LINUXIUM:~$

Next you can search through the 'apt' history files looking for entries showing an application installation:


See how the highlighted entry shows the package installed together with the dependent packages that were automatically installed at the same time.

However this will not show any packages that were installed using 'dpkg' or by a tool like 'gdebi'. To see all the packages we can look at '/var/log/dpkg.log' however the 'package -> dependent package' relationship is somewhat lost:


Finally these logs do not last for ever as they are rotated to save space as defined by their respective configuration files:


So one solution to the problem of creating a duplicate test environment would be to use 'isocomparepkgs.sh' to establish which packages were locally installed and not part of the original ISO and then respin the original ISO adding these packages. Except that in a more complex environment this can become a somewhat unrealistic alternative :


Which finally leads me to my other tool: 'isomimicpkgs.sh'.

        linuxium@LINUXIUM:~$ isomimicpkgs.sh
        Usage: isomimicpkgs.sh <target ISO>
   
        or: isomimicpkgs.sh <source ISO> <target ISO>
        linuxium@LINUXIUM:~$

The usage syntax is straightforward: you either run the script against two ISOs to see how to mimic the package of those installed in the source ISO to the target ISO or you run the script against an ISO to see how to mimic the locally installed packages to that ISO.

In the example we have been using so far you would run the command:
isomimicpkgs.sh ubuntu-18.04-desktop-amd64.iso
The resultant output looks like this:


The tool first identifies those local packages that need to be installed manually as they are not Canonical packages (e.g. 'google-chrome-stable', 'megasync' and 'xnview').

The tool has then simplified the number of packages that were needed to be installed down from 698 to 102 by attemting to reverse engineer the package dependencies and determine a minimum set of packages that obtains the same result.

However as a result it has highlighted some packages that then also need to be removed (or purged), either because they had been previously removed since original installation (for example 'youtube-dl') or due to a dependency issue (in this case it is actually 'php7.2-gd'). Therefore rather than simply removing all the identified additional packages as there could be consequencies because of package dependencies it is advisable to check each one individually first. This is best achieved by respinning the intermediate ISO using the '--interactive' option. For example:


The resultant ISO can then be compared with the locally installed packages to verify that we have successfully mimicked the local environment:


In the final post of this series I will demonstrate the usage of my 'ISO' tools in a real-life example.

The initial release of 'isomimicpkgs.sh' can be downloaded from here.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.


Demonstrating the usage of 'ISO' tools with a real-life example

$
0
0
This is the final of three posts about my 'ISO' tools that include 'isorespin.sh', 'isocomparepkgs.sh' and 'isomimicpkgs.sh' scripts and in it I will demonstrates their usage with a worked example.

The example is about a testing PC that is currently running Ubuntu 18.04 LTS and now that Ubuntu 20.04 LTS has been released I'm considering how to upgrade it.

Normally Ubuntu does not prompt you to upgrade until the first point release becomes available (in this case 20.04.1) so in theory I could use the interim to plan however I want to upgrade now.

I'm considered two approaches to perform the upgrade. The first is the accepted way of upgrading where you simply run a command or use the Ubuntu update manager and upgrade the currently installed Ubuntu (i.e. 18.04 to 20.04). The second approach is arguably more of a migration where you backup your data then install a clean Ubuntu 20.04 and then restore your data which despite semantics is still effectively upgrading from 18.04 to 20.04.

Canonical provide good documentation on how to perform upgrades (e.g. How to upgrade from Ubuntu 18.04 LTS to 20.04 LTS today) however one point that must not be overlooked is whether your upgraded system will still have the same functionality of your existing one.

Like most users besides the applications or packages provided by the base installation I've also added packages and an upgrade will likely affect them. Unfortunately some packages are tied to a release version so maybe unavailable in a new releases or unsupported at best.

When you perform a upgrade traditionally you would remove obsolete packages in order to get a clean system and in part that is why planning for an upgrade is necessary.

The alternative approach of restoring your data after a fresh install also isn't just that simple as besides restoring your data you then need to install all the packages you added to the previous release albeit bearing in mind that not all packages may be available.

One key issue you face with the second approach is knowing what packages you have currently installed and of them which were manually installed subsequent to the initial base installation.

Looking at the second approach first, lets start by identifying the packages that will be required to be additionally installed.

On Debian and other dpkg-based distributions which includes Ubuntu you could try running the following command to gather this information:

grep " install" /var/log/dpkg.log

However this only works if the log file hasn't been rotation in which case you need:

grep " install" /var/log/dpkg.log /var/log/dpkg.log.1

But again this isn't that simple as older log files get compressed and worse still eventually deleted by log rotation (take a look at your '/etc/logrotate.d/dpkg' file) so it can't be used as a way to give you the entire history of your system.

As a simple alternative you can use my 'isocomparepkgs.sh' to compare the packages currently installed with those included in the installation ISO.

First I need to find out which ISO was used for the initial install as looking at '/etc/os-release' will only show the current version (in my case 18.04.4). Looking at '/var/log/installer/media-info' shows the release version (mine was 18.04) and release date (20180426). This indicates that the installation was done from the 'ubuntu-18.04-desktop-amd64.iso' ISO and this can be confirmed by comparing with the ISO's '.disk/info':

if [ "$(cat /var/log/installer/media-info)" == \
     "$(7z x -so ubuntu-18.04-desktop-amd64.iso .disk/info)" ]; then \
        echo is the ISO
else
        echo not the ISO
fi

However since its installation multiple 'apt upgrade' commands have upgraded my system from 18.04 to 18.04.4 so to ensure I make a true comparison I should first use 'isorespin.sh' to upgrade an 18.04 ISO the same way (as remember that point releases also introduce the HWE stack so an upgraded 18.04.4 is not exactly the same as a release 18.04.4 ISO).

Therefore I need to first simulate the elapsed upgrades with:

isorespin.sh -i ubuntu-18.04-desktop-amd64.iso -b GRUB-64 --dist-upgrade


after which I renamed the resultant ISO as 'dist-upgraded-ubuntu-18.04-desktop-amd64.iso' and then compare with my existing system:

isocomparepkgs.sh dist-upgraded-ubuntu-18.04-desktop-amd64.iso

The resultant 'isocomparepkgs.log' contains a list of packages that are only locally installed:


Even now it is not that simple as the list of packages is very long as it includes package dependencies including libraries but at least I have a list which could be used for the second approach.

This is where 'isomimicpkgs.sh' comes in useful. It will take that long list of packages and try and identify which core packages would achieve the same result if installed using 'apt' from Ubuntu sources as well as identifying those which require manual installation. So it is just a simple question of running:

isocomparepkgs.sh dist-upgraded-ubuntu-18.04-desktop-amd64.iso

and the log file shows:


A point to note here is that at some stage during using Ubuntu 18.04 I installed the 'lxde' package and later purged 'youtube-dl'. It also looks like a package dependency changed with respect to PHP as if I follow the suggested purge command exactly then I will also remove a package I want to keep ('phoronix-test-suite') again because of package dependencies (see my documentation on 'isomimicpkg.sh' for more details).

Another observation to make here is that three packages were identified as requiring manually installation.

So after verifying and running the appropriate 'apt install' and 'apt purge' commands I then renamed the resultant ISO as 'localised_18.04.iso' which is an ISO that effectively reflects my current environment (less chome, megasync and xnview).

Therefore if I was to adopt the second approach to upgrading from 18.04 to 20.04 I could use these two 'apt' commands as a basis of what to install over a clean 20.04 installation. Obviously some packages like 'linux-headers-4.15.0-99-generic' are not required whereas others like 'edid-decode' I may decide I no longer require and some packages like 'phoronix-test-suite' are just not available. But at least it is a starting point.

However the first approach was to upgrade 18.04 using the command line. Now that I have an ISO that mimics my current environment I can install this on a separate test box, run the upgrade and observe what happens without the need of lengthy data backups or restores. This way I can see what happens during the upgrade.

Or I can manually upgrade this ISO. However because of 'snaps' not working in an 'chroot' environment the upgrade is slightly more convoluted.

First I need to replace the 'bionic' repositories with the 'focal' ones:

isorespin.sh -i localised_18.04.iso -b GRUB-64 -r "--remove deb http://archive.ubuntu.com/ubuntu/ bionic main restricted" -r "--remove deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted" -r "--remove deb http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted" -r "deb http://archive.ubuntu.com/ubuntu/ focal main restricted" -r "deb http://security.ubuntu.com/ubuntu/ focal-security main restricted" -r "deb http://archive.ubuntu.com/ubuntu/ focal-updates main restricted"

after which I rename the resultant ISO as 'interim_first_localised_pseudo_20.04.iso'. Then in theory I could perform a 'dist-upgrade' on this ISO however this fails due to a problem with the 'texlive-binaries' package. So instead I purge that package and then perform the upgrade:

isorespin.sh -i interim_first_localised_pseudo_20.04.iso -b GRUB-64 -e texlive-binaries --dist-upgrade

creating a new ISO I call 'interim_second_localised_pseudo_20.04.iso' and renaming the log file as 'interim_second_localised_pseudo_20.04.iso.isorespin.log'. I then add back in packages removed as a result of '-e texlive-binaries' with:

isorespin.sh -i interim_second_localised_pseudo_20.04.iso -b GRUB-64 -p '"'$(sed -n '/The following packages will be REMOVED/,/The following NEW packages will be installed/p;/The following NEW packages will be installed/q' interim_second_localised_pseudo_20.04.iso.isorespin.log | sed '1d;$d;s/\*//g' | xargs | sed 's/libsynctex1 //' | sed 's/libtexlua52 //')'"' --debug

which is a command derived through trial and error as a couple of the packages must be excluded in order to successfully add back the packages. I rename the final ISO as 'localised_pseudo_20.04.iso'.

At this point I can now mimic a standard Ubuntu 20.04 ISO from this localised pseudo 20.04:

isomimicpkgs.sh localised_pseudo_20.04.iso ubuntu-20.04-desktop-amd64.iso

to give me the 'apt' commands to create a localised Ubuntu 20.04 ISO from the localised pseudo Ubuntu 20.04 ISO:

$(grep "^isorespin.sh" isomimicpkgs.log | head -1) --debug
isorespin.sh -i linuxium-ubuntu-20.04-desktop-amd64.iso -b GRUB-64 -e "libxfce4util-bin python3-pyxattr rtmpdump youtube-dl"

which I triumphantly name 'localised_20.04_from_pseudo_20.04.iso'.

Finally I can compare this 'localised_20.04_from_pseudo_20.04.iso' with the earlier 'localised_pseudo_20.04.iso' to shows me the new packages installed as part of Ubuntu 20.04 and the packages I loose as part of the upgrade:


and importantly for me this includes loosing 'phoronix-test-suite'.

This still leaves me the option of manually installing my mimic'ed Ubuntu 18.04 testing environment and performing the upgrade the official Ubuntu way.  However before doing this I want to make sure that I create the most similar environment as possible prior to the upgrade so first I capture the full list of installed packages on Ubuntu 18.04 with:

dpkg-query -Wf '${db:Status-Abbrev}${Package}\n' | grep '^ii' | awk '{print $2}' | sort > source_locally_installed_packages.18.04

Now I can install my 'localised_18.04.iso' on the separate test box. Once installed I need to examine the installation log as 'ubiquity' (Ubuntu's installer) removes some packages it believes are not required:

grep -i removed /var/log/installer/syslog

and I need to re-install them with:

sudo apt install apt-clone cifs-utils dmeventd dpkg-repack gparted grub-efi-amd64 kpartx lvm2 python-crypto python-ldb python-samba python-tdb qt5-gtk-platformtheme qttranslations5-l10n samba-common samba-common-bin

I also need to remove any packages added by 'ubiquity' so I compare the output of running a 'dpkg-query' with the previous 'source_locally_installed_packages.18.04' output and see that I need to:

sudo apt purge grub-pc-bin

As a check I can then compare the 'localised_18.04.iso' with newly locally installed packages using:

isocomparepkgs.sh localised_18.04.iso

which confirms that I have successfully replicated the environment:



Now I can perform the upgrade with

(Alt+F2) update-manager -c -d



As I perform the upgrade I am given the opportunity to monitor what will happen:



and make important decisions:



Once complete I can then mimic the resultant environment to a standard Ubuntu 20.04 ISO with:

isomimicpkgs.sh ubuntu-20.04-desktop-amd64.iso

creating a final 'localised_20.04.iso'.

I can then compare this ISO with the ones created earlier (i.e. 'localised_pseudo_20.04.iso' and 'localised_20.04_from_pseudo_20.04.iso') and see more implications and effects of upgrading for use in my final planning of which approach to use and what information I need to successfully upgrade my testing PC from Ubuntu 18.04 to 20.04.

Hopefully this worked example show the usefulness of my 'ISO' tools and how they can be applied to everyday problems rather than just being used to boot Ubuntu on a 32-bit device.

Please donate if you find the scripts useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.

Canonical announces new point releases - Ubuntu 20.04.1 and 18.04.5

$
0
0

Canonical have released both the first point release of Ubuntu 20.04 Long-Term Support (LTS) as Ubuntu 20.04.1 and the fifth point release of Ubuntu 18.04 Long-Term Support (LTS) as Ubuntu 18.04.5.

I’ve respun the desktop ISOs using my ‘isorespin.sh‘ script and created ISOs suitable for Intel Atom and Intel Apollo Lake devices:

Atom (-i ubuntu-20.04.1-desktop-amd64.iso --atom)
Apollo (-i ubuntu-20.04.1-desktop-amd64.iso --apollo)
Atom (-i ubuntu-18.04.5-desktop-amd64.iso --atom)
Apollo (-i ubuntu-18.04.5-desktop-amd64.iso --apollo)

I've also respun the 'Focal Fossa' desktop ISO with the '--server' option to create a pseudo server ISO suitable for Intel devices with a 32-bit bootloader:

Server (-i ubuntu-20.04.1-desktop-amd64.iso --server)

Also announced are the official 20.04.1 flavours of Ubuntu including Lubuntu which I've also respun to created an ISO suitable for Intel Atom devices:

Atom (-i lubuntu-20.04.1-desktop-amd64.iso --atom)


Downloading Note

After downloading an ISO file it is recommended to test that the file is correct and safe to use by verifying the integrity of the downloaded file. An error during the download could result in a corrupted file and trigger random issues during the usage of the ISO.

The program 'md5sum' is designed to verify data integrity using the MD5 (Message-Digest algorithm 5) 128-bit cryptographic hash. The MD5 calculation gives a checksum (called a hash value), which must equal the MD5 value of a correct ISO.

First open a terminal and go to the correct directory to check a downloaded ISO. Then run the command 'md5sum <ISO>' for example:
md5sum linuxium-atom-ubuntu-20.04.1-desktop-amd64.iso
'md5sum' should then print out a single line after calculating the hash:

5157b92b64ac5a9a0b69c8d27888c739  linuxium-atom-ubuntu-20.04.1-desktop-amd64.iso

Compare the hash (the alphanumeric string on left) from your output with the corresponding hash below. If both hashes match exactly then the downloaded file is almost certainly intact. However if the hashes do not match then there was a problem with the download and you should download the file again.


ISO 'md5sum' hashes

5157b92b64ac5a9a0b69c8d27888c739  linuxium-atom-ubuntu-20.04.1-desktop-amd64.iso
58b349bc95ac9f545a9480eda410b9f1  linuxium-apollo-ubuntu-20.04.1-desktop-amd64.iso
9b460cbc70020f117217bf96385d7a3f  linuxium-atom-ubuntu-18.04.5-desktop-amd64.iso
8231e6792cc3c8eed61dbe9b47563dc4  linuxium-apollo-ubuntu-18.04.5-desktop-amd64.iso
58b65aca1795562cf16471a45cdf35c8  linuxium-ubuntu-20.04.1-server-amd64.iso
d7fb5c40db10b100b2ad0428a1ff0dcf  linuxium-atom-lubuntu-20.04.1-desktop-amd64.iso


Please donate if you find these ISOs useful.

Intel NUC 9 Extreme “Ghost Canyon” Kit – NUC9i9QNX Review

'BootHole' implications for 'isorespin.sh'

$
0
0

 

(Credit: https://eclypsium.com/2020/07/29/theres-a-hole-in-the-boot)

When it was discovered that GRUB2 contained various vulnerabilities that would allow UEFI Secure Boot to be bypassed and which became known as the “BootHole” vulnerability (CVE-2020-10713), the recommendation was that all operating systems using GRUB2 with Secure Boot must release new installers and bootloaders. 

I reviewed 'isorespin.sh' at that time as one of it's key features is the option to add a GRUB2 bootloader to allow ISOs to boot on the many Intel devices limited by their BIOS requiring a 32-bit bootloader to boot a 64-bit OS.

My initial 'fix' was based around Ubuntu's response by recompiling and adding the latest fixed GRUB2 bootloader from 'groovy' (Ubuntu 20.10) and let the Ubuntu package manager 'apt' install the appropriate GRUB2 binaries to the ISO whilst being respun.

This initially worked, however after receiving what can only be described as some abusive 'hate' email from a user complaining that 'isorespin.sh' fails when installing the 32-bit binaries, I investigated and found that Canonical had effectively removed the earlier 32-bit GRUB2 packages with vulnerabilities.

The original 'isorespin.sh' process was to download the 32-bit GRUB2 packages whose version matched the 64-bit GRUB2 packages in the ISO and update the relevant package file with the details of these packages. However in Canonical's process when a package is replaced by a newer version at some point older versions get archived so the 'isorespin.sh' download process needs to perform the download from the archive location. At this point the package information is still typically available in the package manager's cache so it is still possible to update the relevant package file.

But in order to add the other functionality in 'isorespin.sh' such as updating the kernel or installing a package as part of respinning an ISO it is also necessary to update the package manager's cache. The issue that "BootHole" subsequently created for 'isorespin.sh' was that because the cache was now updated, the earlier versions of the GRUB2 packages with the vulnerabilities were (obviously) no longer included to prevent them from being selected and installed. The consequence was that because the downloaded earlier versioned 32-bit GRUB2 packages were no longer supported, when they were further processed either by 'isorespin.sh' or as part of ISO installation, errors occurred.

Part of the problem in fixing these errors was wanting to mimic the original ISO's ability to be installed either with or without a network connection and also address the "BootHole" vulnerability as part of respinning the ISO. A new issue was encountered because by simply downloading the latest and therefore fixed 32-bit GRUB2 packages left their package dependencies untouched. This leads to package incompatibility when trying to install these later versioned packages.  

To address this I've made the decision to continue to download the 32-bit GRUB2 packages whose version matches that of the ISO thereby keeping the integrity of the ISO. However in recognising that any package in the ISO's pool structure could be superseded by security updates I also then ensure that all of the pool packages are updated to their respective current version at time of respinning the ISO. This also means that their versions are reflected in the ISO's package manager's cache. Finally to correct the GRUB2 package dependencies I also update any GRUB2 packages currently installed in the ISO's filesystem.

Whilst this addresses the vulnerabilities caused by "BootHole" it does mean that if the Ubiquity installer installs other packages from the pool structure it may still result in package dependency issues. The workaround if this occurs is to either individually update the affected packages when respinning the ISO or use the '--dist-upgrade' option to upgrade all installed packages.

This newest version (8.6.4) is now available from 'isorespin.sh'. 

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.


Canonical have announced a new point release for Ubuntu 16.04 LTS - 16.04.7 (Xenial Xerus)

$
0
0
Canonical have released the sixth point release of Ubuntu 16.04 Long-Term Support (LTS) as Ubuntu 16.04.7.

I’ve respun the desktop ISO using my ‘isorespin.sh‘ script and created ISOs suitable for Intel Atom and Intel Apollo Lake devices:

Atom (-i ubuntu-16.04.7-desktop-amd64.iso --atom)
Apollo (-i ubuntu-16.04.7-desktop-amd64.iso --apollo)


Downloading Note

After downloading an ISO file it is recommended to test that the file is correct and safe to use by verifying the integrity of the downloaded file. An error during the download could result in a corrupted file and trigger random issues during the usage of the ISO.

The program 'md5sum' is designed to verify data integrity using the MD5 (Message-Digest algorithm 5) 128-bit cryptographic hash. The MD5 calculation gives a checksum (called a hash value), which must equal the MD5 value of a correct ISO.

First open a terminal and go to the correct directory to check a downloaded ISO. Then run the command 'md5sum <ISO>' for example:
md5sum linuxium-atom-ubuntu-16.04.7-desktop-amd64.iso
'md5sum' should then print out a single line after calculating the hash:

e1c5c463c3d2078f7a26d65472b59973  linuxium-atom-ubuntu-16.04.7-desktop-amd64.iso

Compare the hash (the alphanumeric string on left) from your output with the corresponding hash below. If both hashes match exactly then the downloaded file is almost certainly intact. However if the hashes do not match then there was a problem with the download and you should download the file again.


ISO 'md5sum' hashes

e1c5c463c3d2078f7a26d65472b59973  linuxium-atom-ubuntu-16.04.7-desktop-amd64.iso
ee3367e767d2c0938cc12776d5cf288d  linuxium-apollo-ubuntu-16.04.7-desktop-amd64.iso


Please donate if you find these ISOs useful.

Respun ISOs Questionnaire

$
0
0

I've just released a new version of 'isorespin.sh' that supports the respinning of the latest Ubuntu and Ubuntu flavoured 20.10 (Groovy Gorilla) ISOs.

However I don't have sufficient space available at the moment to post an example ISO similar to those posted here.

So I've created a questionnaire to ask which ISOs are required both now and in the future.

There are only three sections:

Types of ISOs
Distro releases
Future ISOs

containing a total of 10 questions requiring a simple 'yes' or 'no' answer and a final open-ended question.

Please complete the questionnaire to ensure your opinion and needs are heard.

Also if you find the script or ISOs useful please donate using the following link http://goo.gl/nXWSGf as everything helps with development costs.


Virtualization Performance on an Intel NUC 11 Enthusiast Phantom Canyon NUC11PHKi7C

$
0
0

I've previously looked at Windows and Linux performance on the NUC11PHKi7C Enthusiast Phantom Canyon which is Intel’s latest NUC 11 flagship product specifically targeting gamers as it includes an NVIDIA RTX 2060 GPU.

One usage aspect I didn't test was virtualization and this brief article looks at the performance running VirtualBox and WSL2 on the NUC11PHKi7C and comparing it to Intel’s previous NUC with a discrete GPU: the NUC 9 Extreme Ghost Canyon.


Hardware Overview

For the NUC 9 Extreme I’ve using a NUC9i7QNX model and I purchased both the NUC11PHKi7C and NUC9i7QNX as barebone devices.

The NUC11PHKi7C has an Intel Core i7-1165G7 Tiger Lake processor which is a quad-core 8-thread 2.80 GHz processor boosting to 4.70 GHz and also includes an NVIDIA N18E-G1-B notebook graphics card which is a GeForce RTX 2060 mobile GPU. I’ve installed a 2TB M.2 2280 NVMe drive from addlink (S70) and 64GB (2 x 32GB) DDR4 3200MHz memory from G.SKILL.

The NUC9i7QNX has an Intel Core i7-9750H Coffee Lake processor which is a hex-core 12-thread 2.60 GHz processor boosting to 4.50 GHz. I've installed a 2TB M.2 2280 NVMe drive from ADATA (XPG 8200 Pro), 64GB (2 x 32GB) of Team Group’s Team Elite DDR4 3200MHz memory and an EVGA GeForce RTX 2060 KO ULTRA GAMING GPU.


Software Overview

On each device I've installed Windows 10 Pro and Ubuntu 20.04 LTS as dual boot. On Windows I've enabled Windows Subsystem for Linux (WSL) version 2 and then installed Ubuntu 20.04 LTS Linux distribution for WSL. Then for each OS I've installed Oracle VM VirtualBox and created VMs of either Windows 10 Enterprise or Ubuntu 20.04 LTS as antithesis to the host OS. 


Installation Issues

Whilst there were no instalation problems with the NUC9i7QNX, the NUC11PHKi7C encountered a major issue. Initially for Ubuntu I was using the latest kernel (5.8.0-48-generic). Once Windows 10 Enterprise was installed in VirtualBox I noticed the VM occationally crashing for no apparent reason. However after downloading Passmark Performance Test version 10.1 the installation file refused to run:

I then had slightly more success in downloading and installing Passmark Performance Test version 9.0 however the application then refused to run:

but I did get Passmark Performance Test version 8.0 to both install and run:

however it subsequently crashed the VM.

After many reinstalls and web searching I discoverd that a bug for the crashing issue has already been raised: https://www.virtualbox.org/ticket/20180 and that 'using a Linux kernel 5.4 does not exhibit the problem'. Switching to the 5.4.0-70-generic kernel did indeed solve all the problems including no more crashes and allowed the successful installation and execution of Passmark Performance Test version 10.1.


Virtualization on Windows

For a Windows baseline I ran the CPU tests from Passmark Performance Test natively in Windows:

NUC9i7QNX

NUC11PHKi7C

and interestingly despite having fewer cores the NUC11PHKi7C's Windows performance was 3% better than on the NUC9i7QNX.

The first virtualization comparison is against running Windows in VirtualBox on Ubuntu where I ran the same CPU tests using the Linux version of Passmark Performance Test:

NUC9i7QNX

NUC11PHKi7C

and this shows that hardware-wise the NUC11PHKi7C performance was 18% worse than the NUC9i7QNX. Software-wise virtualization on the NUC9i7QNX performed similarly to its native performance at only 1% lower however for the NUC11PHKi7C it was 19% worse.

For an Ubuntu baseline I ran the CPU tests from Passmark Performance Test Linux natively in Ubuntu:

NUC9i7QNX

NUC11PHKi7C

and this time the NUC9i7QNX Ubuntu performance was 1% better than on the NUC11PHKi7C.

The second virtualization comparison is against running Ubuntu in VirtualBox on Windows:

NUC9i7QNX

NUC11PHKi7C

which again hardware-wise shows the NUC11PHKi7C performing worse than the NUC9i7QNX but this time by only 12%. Virtualization however is markedly different with the NUC11PHKi7C being 47% worse than running Ubuntu natively and 42% worse for the NUC9i7QNX.

The final virtualization comparison is against WSL2:

NUC9i7QNX

NUC11PHKi7C

where the NUC11PHKi7C performed 4% better than the NUC9i7QNX hardware-wise. It was also the best for Ubuntu virtualisation with only a loss of 1% for the NUC11PHKi7C and a 6% loss for the NUC9i7QNX. It should also be pointed out that all of the results can be affected by test run margin of error.

The full results are summarised below:



Conclusion

Although this is very limited testing it suggests that from a hardware perspective VirtualBox on the 6-core 12-thread NUC9i7QNX performs better than on the 4-core 8-thread NUC11PHKi7C even though the native performance is similar. Virtualbox on Windows is much worse than on Ubuntu however the real winner is the performance of running Ubuntu under WSL2 as it is comparable to the native performance. Also of note is that Ubuntu performance is slightly better than Windows performance.


Donate

Please donate if you find these types of comparisons useful using the following link http://goo.gl/nXWSGf as everything helps with hardware costs.

Canonical have announced the release of Ubuntu 21.04 (Hirsute Hippo)

$
0
0

 


Canonical have announced the latest release of Ubuntu 21.04 (Hirsute Hippo).

I’ve respun the desktop ISO using my ‘isorespin.sh‘ script and created ISOs suitable for Intel Atom and Intel Apollo Lake devices:

Atom (-i ubuntu-21.04-desktop-amd64.iso --atom)
Apollo (-i ubuntu-21.04-desktop-amd64.iso --apollo)



Also announced are the official 21.04 flavours of Ubuntu including Lubuntu which I've also respun to created an ISO suitable for Intel Atom devices:

Atom (-i lubuntu-21.04-desktop-amd64.iso --atom)


Downloading Note

After downloading an ISO file it is recommended to test that the file is correct and safe to use by verifying the integrity of the downloaded file. An error during the download could result in a corrupted file and trigger random issues during the usage of the ISO.

The program 'md5sum' is designed to verify data integrity using the MD5 (Message-Digest algorithm 5) 128-bit cryptographic hash. The MD5 calculation gives a checksum (called a hash value), which must equal the MD5 value of a correct ISO.

First open a terminal and go to the correct directory to check a downloaded ISO. Then run the command 'md5sum <ISO>' for example:

md5sum linuxium-atom-ubuntu-21.04-desktop-amd64.iso

'md5sum' should then print out a single line after calculating the hash:

2a16ded012decf9c5533692b7d0231f9  linuxium-atom-ubuntu-21.04-desktop-amd64.iso

Compare the hash (the alphanumeric string on left) from your output with the corresponding hash below. If both hashes match exactly then the downloaded file is almost certainly intact. However if the hashes do not match then there was a problem with the download and you should download the file again.


ISO 'md5sum' hashes

2a16ded012decf9c5533692b7d0231f9  linuxium-atom-ubuntu-21.04-desktop-amd64.iso
f131fab9d8ac7692923641dcd1f6c4e0  linuxium-apollo-ubuntu-21.04-desktop-amd64.iso
dc26eca9db2f10938090baeb4ea7d1fd  linuxium-atom-lubuntu-21.04-desktop-amd64.iso


Please donate if you find these ISOs useful.

Canonical announces new point releases - Ubuntu 20.04.3 and 18.04.6

$
0
0

Canonical have released both the third point release of Ubuntu 20.04 Long-Term Support (LTS) as Ubuntu 20.04.3 and an unexpected six point release of Ubuntu 18.04 Long-Term Support (LTS) as Ubuntu 18.04.6 as a result of GRUB2 Secure Boot Bypass 2021.

I’ve respun the desktop ISOs using my ‘isorespin.sh‘ script and created ISOs suitable for Intel Atom and Intel Apollo Lake devices:

Atom (-i ubuntu-20.04.3-desktop-amd64.iso --atom)
Apollo (-i ubuntu-20.04.3-desktop-amd64.iso --apollo)
Atom (-i ubuntu-18.04.6-desktop-amd64.iso --atom)
Apollo (-i ubuntu-18.04.6-desktop-amd64.iso --apollo)

I've also respun the 'Focal Fossa' desktop ISO with the '--server' option to create a pseudo server ISO suitable for Intel devices with a 32-bit bootloader:

Server (-i ubuntu-20.04.3-desktop-amd64.iso --server)

Also announced are the official 20.04.3 flavours of Ubuntu including Lubuntu which I've also respun to created an ISO suitable for Intel Atom devices:

Atom (-i lubuntu-20.04.3-desktop-amd64.iso --atom)


Downloading Note

After downloading an ISO file it is recommended to test that the file is correct and safe to use by verifying the integrity of the downloaded file. An error during the download could result in a corrupted file and trigger random issues during the usage of the ISO.

The program 'md5sum' is designed to verify data integrity using the MD5 (Message-Digest algorithm 5) 128-bit cryptographic hash. The MD5 calculation gives a checksum (called a hash value), which must equal the MD5 value of a correct ISO.

First open a terminal and go to the correct directory to check a downloaded ISO. Then run the command 'md5sum <ISO>' for example:

md5sum linuxium-atom-ubuntu-20.04.3-desktop-amd64.iso

'md5sum' should then print out a single line after calculating the hash:

166bef608b7cb64dd92ba804c490fa9e linuxium-atom-ubuntu-20.04.3-desktop-amd64.iso

Compare the hash (the alphanumeric string on left) from your output with the corresponding hash below. If both hashes match exactly then the downloaded file is almost certainly intact. However if the hashes do not match then there was a problem with the download and you should download the file again.


ISO 'md5sum' hashes

e2ec97be8ed27967335174e5551f29ce linuxium-atom-ubuntu-18.04.6-desktop-amd64.iso
ca7634b2e5c7d7ac8885b13a491242f9 linuxium-apollo-ubuntu-18.04.6-desktop-amd64.iso
166bef608b7cb64dd92ba804c490fa9e linuxium-atom-ubuntu-20.04.3-desktop-amd64.iso
1c0a56d3a7806c92f9c3ba0104ed4a1d linuxium-apollo-ubuntu-20.04.3-desktop-amd64.iso
91a6ac93e8f5976b73ee6c90ea4aacc9 linuxium-ubuntu-20.04.3-server-amd64.iso
052e5d0ab5e1b997b4df76d64c3db5a6 linuxium-atom-lubuntu-20.04.3-desktop-amd64.iso


Please donate if you find these ISOs useful.

New release of 'isorespin.sh'

$
0
0

Following news of the GRUB2 Secure Boot Bypass 2021 and as a result of Google's security changes on Google Drive together with the first daily build's from Canonical of Ubuntu 21.10 (impish) and point releases for 20.04.3 and 18.04.6 I've updated my ‘isorespin.sh‘ script and respun some ISOs suitable for Intel Atom and Intel Apollo Lake devices.


Note that support for 21.10 (impish) is not finalized as the release is still under development so respinning will be experimental at this stage. However due to a new compression tool being used an additional package 'zstd' will need to be installed prior to attempting any respinning.


Unfortunately interest seems to have declined judging by the lack of donations so please remember to donate if you find this work useful.


Supporting 'impish' releases

$
0
0

Now that Canonical has released Ubuntu 21.10 (Impish Indri) you can use my latest release of ‘isorespin.sh‘ to respin Ubuntu ISOs:

and Lubuntu ISOs:

as well as the other supported flavours but remember to install the additional package 'zstd' before respinning as this release uses a new compression tool.

Please remember to donate if you find my work useful.

First look at SteamOS 3

$
0
0

 


Having downloaded the Steam Deck recovery image and written to a USB using 'Rufus', I was surprised that booting on various AMD mini PCs has so far been unsuccessful whereas after just a few tweaks it successfully booted on a couple of different Intel NUCs. 

Installation was easy however rebooting again needed tweaks to get to a SteamOS 3 desktop.

Package installation was hindered by trust issues related to signatures but eventually I was able to install 'neofetch' as an example.

However 'gaming' is limited as most games baulk at the lack of a suitable graphics capability and/or drivers. Time for some further investigating!

  

If things don't change, things will stay as they are.

$
0
0

 


The 'canary' ISO for Ubuntu 22.04 (Jammy Jellyfish) introduces the new Ubuntu Desktop installer which uses 'subiquity' as a backend and 'Flutter' for the UI.

Rather than hack 'isorespin.sh' to coerce compatibility I'm developing a new script which takes the most useful features and targets functionality that is more relevent to current usage.

Despite plus ça change, plus c'est la même chose (the more things change, the more they stay the same) I thought it best to give the script a new name: 'isorespinner.sh'.

More to come.


“If things don't change, things will stay as they are.” - John Englander 

"Plus ça change, plus c'est la même chose." - Jean-Baptiste Alphonse Karr


Canonical have announced the release of Ubuntu 22.04 LTS (Jammy Jellyfish)

$
0
0
For the last few years I've respun various ISOs to support running Ubuntu, Ubuntu flavours and Ubuntu-based distros on mini PCs by specifically addressing the issue of those devices that use restrictive bootloaders including 32-bit GRUB.

I've relied on donations to support my work and cover development and storage costs. Recently these have been few and far between indicating that the demand for such ISOs is not there.
As a result I'm reducing the number of ISOs I provide and only offer some examples of how my 'isorespinner.sh' script works.




With Canonical announcing the latest release of Ubuntu 22.04 LTS (Jammy Jellyfish) I’ve respun the desktop ISO to create an ISO which includes support for Intel Atom devices as well as any other 64-bit (x86-64/AMD64) PC:

linuxium-ubuntu-22.04.4-desktop-amd64.iso (-i ubuntu-22.04-desktop-amd64.iso -b GRUB-32 -l rtl8723bs_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -f linuxium-install-broadcom-drivers.sh -f wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh)




I've also respun Lubuntu:

linuxium-lubuntu-22.04.4-desktop-amd64.iso (-i lubuntu-22.04-desktop-amd64.iso -b GRUB-32 -l rtl8723bs_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -f linuxium-install-broadcom-drivers.sh -f wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh)




Finally for the adventurous few wanting to try the new Ubuntu desktop installer I've respun the desktop ISO (jammy-desktop-canary-amd64.iso version 20220418.1) with a set of options I found prevent​ed​ ​both ​freezing during installation and ​also ​running out of memory when creating a new 'initramfs' on Intel Atom devices:

linuxium-20220418.1-ubuntu-jammy-desktop-canary-amd64.iso (-i jammy-desktop-canary-amd64.iso -b GRUB-32 -g intel_idle.max_cstate=1 -g fsck.mode=skip -p lz4 -c "sed -i 's/^COMPRESS=zstd/COMPRESS=lz4/' /etc/initramfs-tools/initramfs.conf")


Downloading Note

After downloading an ISO file it is recommended to test that the file is correct and safe to use by verifying the integrity of the downloaded file. An error during the download could result in a corrupted file and trigger random issues during the usage of the ISO.

The program 'md5sum' is designed to verify data integrity using the MD5 (Message-Digest algorithm 5) 128-bit cryptographic hash. The MD5 calculation gives a checksum (called a hash value), which must equal the MD5 value of a correct ISO.

First open a terminal and go to the correct directory to check a downloaded ISO. Then run the command 'md5sum <ISO>' for example:

md5sum linuxium-ubuntu-22.04.4-desktop-amd64.iso

'md5sum' should then print out a single line after calculating the hash:

db4bc3918a95f54d6374f06d6d09316c  linuxium-ubuntu-22.04-desktop-amd64.iso

Compare the hash (the alphanumeric string on left) from your output with the corresponding hash below. If both hashes match exactly then the downloaded file is almost certainly intact. However if the hashes do not match then there was a problem with the download and you should download the file again.

ISO 'md5sum' hashes:

db4bc3918a95f54d6374f06d6d09316c  linuxium-ubuntu-22.04-desktop-amd64.iso
4743a3fb5fddfef89c981fcbfac7aee0  linuxium-lubuntu-22.04-desktop-amd64.iso
2512e8241cee853472bbad639b2cbc6e  linuxium-20220418.1-ubuntu-jammy-desktop-canary-amd64.iso


Please donate if you find these ISOs useful.

Customizing Ubuntu ISOs: Documentation and examples of how to use 'isorespinner.sh'

$
0
0

 


isorespinner.sh

This script is a ​progression of 'isorespin.sh'. Whilst 'isorespin' was created to support Ubuntu and similar Linux distributions on mini PCs, some of the functionality that was developed is now unused. Additionally with ​Canonical now trialling a new ​Ubuntu ​desktop installer and the ISO using a new multi-layer filesystem, rather than hack the ​earlier code to coerce compatibility I'​ve​ developed a new script ​which I've called ​'isorespinner.sh'​and provides the most useful features and ​addresses the functionality that is ​now relevant to​ ​respinning.

The syntax of the new script is as follows:

Usage: isorespinner.sh [ -h | -v | --check ]
       isorespinner.sh -i <ISO> [ -w <directory> | -u | -k <kernel> | --dist-upgrade | --upgrade | --interactive | --all | --disk | --debug | ...
       isorespinner.sh ... -b [ GRUB-32 | GRUB-64 ] | -g [ "" | "<kernel boot parameter> ... " ] | c "<cmd> ... " | ...
       isorespinner.sh ... --key "<repo> ... " | -r "<repo> ... " | -p "<pkg> ... " | -l "<pkg.deb> ... " | -e "<pkg> ... " | -f [ "<file> | <directory> ... " ]

with a lot of the options unchanged from their previous functionality as described here.

The primary significant difference between 'isorespin.sh' and 'isorespinner.sh' is that the new script no longer adds a GRUB 32-bit bootloader by default so ​the option '-b GRUB-32'​should be used ​if ​it is ​required. Secondly along with removing redundant options, the GUI interface has been dropped so options can only be specified using the command line​. ​As a result ​all ​the previous functionality can be replicated using the current options. The prerequisite dependencies are the same as before and if not installed the script will identify any that are required.

The new script includes two new features​:

  1. The script tries to run using memory for the temporary files to alleviate continuous read/writes to storage ​and thereby​ reduce drive wear. ​However to exclude running in memory​ ​the option '--disk'​should be specified ​especially if using the script for large and complex respins as currently it is possible to exceed the initial memory allocation ​which ​results ​in running out of 'virtual' space. 
  2. ​The other key new feature is the support for multi-layer filesystem ISOs such as the new canary Ubuntu Jammy 22.04 ISO. However due to the nature of how such ISOs uses layers, respinning becomes somewhat dependent and/or specific to the individual layers​.​ The default language layer is derived from the 'LANG' shell variable or ​set ​as 'English' if either ​the variable is unset or set to 'C.UTF-8'. A different language layer, assuming it is already​ ​supported​ by the ISO​, can be respun by prefixing the respin command with the required 'LANG' variable, so for Spanish for example, use 'LANG=es isorespinner.sh -i ...​'​. If all language layers require respinning​ then ​use the option '--all'. Currently only a 'normal' installation is supported as the 'minimal' installation has not ​yet ​been ​investigated.

Looking at invocation, a good example of why 'isorespinner.sh' is required is the back-porting of the Intel P-State driver fix for Intel Alder Lake processors. Currently the brand new Jammy ISO ships with Linux kernel version 5.15.25 which results in poor performance on Alder Lake due to ITMT support for with P and E core selection. However the Linux kernel version 5.15.35 includes this back-ported fix and this can be incorporated into a respun ISO by using the option '-k v5.15.35' which will then use the Canonical build of the Mainline Test tree prior to the kernel filtering through the kernel rollout process.

Whilst before it was possible to respin an ISO suitable for Intel Atom processors using the single '--atom' option, now the various components have to be first downloaded and respun as specific options e.g. '-b GRUB-32 -l rtl8723bs_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -f linuxium-install-broadcom-drivers.sh -f wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh' as the '--atom' option has been depreciated. For reference these are the 'atom' files:

rtl8723bt_4.12.0_amd64.deb (used when the linux-firmware package is at least version 1.169)
rtl8723bs_4.12.0_amd64.deb (used for earlier versions of the linux-firmware package)
linuxium-install-broadcom-drivers.sh
linuxium-install-UCM2-files.sh (used for ISOs 20.04 and up - rename as 'UCM' without the '2')

I​'ve​ also found that the new multi-layer filesystem ​ISO ​was too 'heavy' for very low-powered Intel mini PCs​. However with a minimum configuration of a Cherry Trail processor ​and 2GB memory​ and some​ specific respinning, both usage and installation was possible. Even then ​an Intel Compute Stick struggled ​but using ​a respun ISO with ​the options '-b GRUB-32 -g intel_idle.max_cstate=1 -g fsck.mode=skip -p lz4 -c "sed -i 's/^COMPRESS=zstd/COMPRESS=lz4/' /etc/initramfs-tools/initramfs.conf"' prevent​ed​ ​both ​freezing during installation and ​also ​running out of memory when creating a new 'initramfs'.

As this is the initial release inevitably there will be improvements required so please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.


Adding a 32-bit GRUB bootloader to boot and install ISOs

$
0
0

 

Many distros no longer include both 32-bit and 64-bit bootloader support. Unfortunately some hardware including most based on the relatively recent Intel Atom processor architecture won't boot 'OOTB'. Whilst I wrote 'isorespin.sh' and 'isorespinner.sh' in part to address this issue I restricted their functionality to the Ubuntu 'family' of ISOs for support purposes. One of the most frequest questions I have been asked is 'can you add support for <insert distro> ISOs?'. Unfortunately distros, even those based on or derived from Ubuntu, are often built with different directory structures and packages. As a result writing a script that caters for multiple distros becomes complex and cumbersome. However with Ubuntu looking to move to a new snap-based installer I've revisited the 32-bit boot issue. 

Initially I created a simple script 'treetoobitiso.sh' to just add the 32-bit GRUB bootloader to the Ubuntu 'family' of ISOs. But if a different distro uses a similar file system layout then essentialy there is no reason why it also wouldn't work so I've extended this script to include an '--unsupported' option to allow running, or at least attempt running, with any ISO.

Invocation is straightforward:

linuxium@LINUXIUM:~$ treetoobitiso.sh -h
Usage: treetoobitiso.sh -h | [ --unsupported ] -i <ISO> [ --disk ]
treetoobitiso.sh: Exiting ... ISO not created.
linuxium@LINUXIUM:~$

being derived from 'isorespinner.sh'. 

To create a bootable Ubuntu or similar ISO:

linuxium@LINUXIUM:~$ treetoobitiso.sh -i linuxmint-20.3-cinnamon-64bit.iso
treetoobitiso.sh: Running in RAM ...
treetoobitiso.sh: Extracting treetoobitiso files ...
treetoobitiso.sh: Extracting ISO ...
treetoobitiso.sh: Adding 32-bit GRUB bootloader ...
treetoobitiso.sh: Spinning ISO ...
treetoobitiso.sh: Respun ISO created as 'linuxium-linuxmint-20.3-cinnamon-64bit.iso' ... see logfile 'treetoobitiso.log' for details.
linuxium@LINUXIUM:~$


For other ISOs:

linuxium@LINUXIUM:~$ treetoobitiso.sh -i pop-os_22.04_amd64_intel_4.iso
treetoobitiso.sh: 'pop-os_22.04_amd64_intel_4.iso' must be an Ubuntu, Kubuntu, Lubuntu, Ubuntu Budgie, Ubuntu GNOME, Ubuntu MATE, Xubuntu or Linux Mint desktop ISO.
treetoobitiso.sh: Exiting ... ISO not created.
linuxium@LINUXIUM:~$

use the '--unsupported' option:

linuxium@LINUXIUM:~$ treetoobitiso.sh --unsupported -i pop-os_22.04_amd64_intel_4.iso
treetoobitiso.sh: Running in RAM ...
treetoobitiso.sh: Extracting treetoobitiso files ...
treetoobitiso.sh: Extracting ISO ...
treetoobitiso.sh: Adding 32-bit GRUB bootloader ...
treetoobitiso.sh: Spinning ISO ...
treetoobitiso.sh: Respun ISO created as 'linuxium-pop-os_22.04_amd64_intel_4.iso' ... see logfile 'treetoobitiso.log' for details.
linuxium@LINUXIUM:~$


As long as the ISO has a file system layout that is recognised by the script, it should work otherwise you will be notified:

linuxium@LINUXIUM:~$ treetoobitiso.sh --unsupported -i Fedora-Workstation-Live-x86_64-35-1.2.iso
treetoobitiso.sh: Running in RAM ...
treetoobitiso.sh: Extracting treetoobitiso files ...
treetoobitiso.sh: Extracting ISO ...
treetoobitiso.sh: ISO structure currently not supported.
treetoobitiso.sh: Exiting ... ISO not created.
linuxium@LINUXIUM:~$

Once booted an installation will be dependent on the ISO's installer. Typically it will try to install a 64-bit GRUB bootloader however it may also fail as whilst it recognises that it was booted by a 32-bit bootloader it likely will not have the code to handle such an installation.

So I've created a companion script 'treetoobit.sh' to install a 32-bit GRUB bootloader during an installation attempt and again, whilst the Ubuntu 'family' of ISOs is supported, if the ISO is Debian/Ubuntu based (i.e. uses the 'apt' package manager) then it may also work using the '--unsupported' option.

For Ubuntu or similar ISOs perform the installation and then immediately before closing the final installation 'success' screen run the 'treetoobit.sh' script.


Some ISOs use the same partitions as Ubuntu for installation which simplifies running the script.


Others use a random name for the mount point and may not leave both the 'installation' and 'boot' partitions mounted at the end of the installation. It is best to first run 'lsblk -a' to see the current state and then perform any required mounting manually.



It is possible that an installation will finish leaving no sign of the partitions so both the mount point and paritions have to be set up first prior to running the script.  


Unfortunately not all installations will complete successfully and depending on what the installer still had left to do means there may be additional manual steps required (hence the 'unsupported' caveat). In the case of Debian looking at the Calamares installer indicates that this includes removing the 'live-*' and 'calamares-settings-debian' packages.


Also if the installation does not support the 'apt' package manager then the script will refuse to run.


Finally, and perhaps of interest to anyone believing in 'snap' security or non-repudiation, specifically for the current 'canary' Ubuntu 22.04 ISO which uses the new Ubuntu ​desktop installer, it is necessary to run the 'treetoobit.sh' script before starting the installation. This will inject code into the desktop installer snap to use 32-bit GRUB packages as otherwise the installation will fail and recovery becomes tedious.

Please donate to help support the development if you find the script useful by following the link http://goo.gl/nXWSGf.