It occurs to me that essentially a copy of the contents of the two PI partitions would be enough, together with an install of Raspios,
followed by a copy back to the fresh SD, to clone a setup.

It seems too easy. What am I missing?


--
“It is not the truth of Marxism that explains the willingness of intellectuals to believe it, but the power that it confers on
intellectuals, in their attempts to control the world. And since...it is
futile to reason someone out of a thing that he was not reasoned into,
we can conclude that Marxism owes its remarkable power to survive every criticism to the fact that it is not a truth-directed but a
power-directed system of thought.”
Sir Roger Scruton

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 9/17/23 04:19, The Natural Philosopher wrote:

It occurs to me that essentially a copy of the contents of the two PI partitions would be enough, together with an install of Raspios,
followed by a copy back to the fresh SD, to clone a setup.

It seems too easy. What am I missing?

Raspberry -> Accessories -> SD Card Copier

--

Knute Johnson

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 17/09/2023 21:19, Knute Johnson wrote:

On 9/17/23 04:19, The Natural Philosopher wrote:

It occurs to me that essentially a copy of the contents of the two PI
partitions would be enough, together with an install of Raspios,
followed by a copy back to the fresh SD, to clone a setup.

It seems too easy. What am I missing?

Raspberry -> Accessories -> SD Card Copier

What has a raspberry to do with this?

--
“People believe certain stories because everyone important tells them,
and people tell those stories because everyone important believes them.
Indeed, when a conventional wisdom is at its fullest strength, one’s agreement with that conventional wisdom becomes almost a litmus test of
one’s suitability to be taken seriously.”

Paul Krugman

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

"The Natural Philosopher" <tnp@invalid.invalid> wrote

| > Raspberry -> Accessories -> SD Card Copier
| >
| What has a raspberry to do with this?
|
Start menu. The raspberry icon in lower left.
You didn't say which Pi you have, so you may not
have that icon or the copier software.

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 18/09/2023 14:03, Newyana2 wrote:

"The Natural Philosopher" <tnp@invalid.invalid> wrote

| > Raspberry -> Accessories -> SD Card Copier
| >
| What has a raspberry to do with this?
|
Start menu. The raspberry icon in lower left.

??? Icon? On a headless Pi Zero?

You didn't say which Pi you have, so you may not
have that icon or the copier software.

Pi Zero



--
The lifetime of any political organisation is about three years before
its been subverted by the people it tried to warn you about.

Anon.

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

"The Natural Philosopher" <tnp@invalid.invalid> wrote

| > |
| > Start menu. The raspberry icon in lower left.
|
| ??? Icon? On a headless Pi Zero?
|
Ah. You might have said that in the first place. It
doesn't make much sense to ask for help and then
be difficult with people who try to help. I gather you
only posted to argue.

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 19/09/2023 03:15, Newyana2 wrote:

"The Natural Philosopher" <tnp@invalid.invalid> wrote

| > |
| > Start menu. The raspberry icon in lower left.
|
| ??? Icon? On a headless Pi Zero?
|
Ah. You might have said that in the first place. It
doesn't make much sense to ask for help and then
be difficult with people who try to help. I gather you
only posted to argue.

No. I am trying to establish whether one can, in essence, make a
bootable PI SD card by taking a raw card, creating the two partitions on
it and loading it up with files, or whether there is the equivalent of a
boot sector which would mandate the use of e.g. dd to create it

Naturally all the documentation assumes a windows numpty and shows you
how to stamp the SD card with an image.

You assumed I wanted to clone the card. I don't. I want to back one up
without using DD because, as the concurrent thread has identified, the
PI expands the file system to the whole card and that makes for a big image. I'd rather back it up as files if possible.

If at some future date I need to recreate it, instead of dd-ing the
image, I would prefer to create the two partitions and populate them
from the tarball.



--
There is nothing a fleet of dispatchable nuclear power plants cannot do
that cannot be done worse and more expensively and with higher carbon
emissions and more adverse environmental impact by adding intermittent renewable energy.

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

The Natural Philosopher <tnp@invalid.invalid> wrote:

No. I am trying to establish whether one can, in essence, make a
bootable PI SD card by taking a raw card, creating the two partitions on
it and loading it up with files,

Yes. In fact if you set the OS up from scratch you don't even need
two partitions, just one FAT32 parition with the boot files
required by the Pi.

I set up a Linux system to boot off the FAT32 parition then run in
a tmpfs. So instead of an image file there's just a ZIP archive
which needs to be unpacked to the root of a FAT32-formatted SD
card. No image writer tool required, and no worries about partition
sizes.

The Pi doesn't need anything like a boot sector. The firmware is
smart enough to find the boot files on the filesystem, provided
it's FAT32.

--
__ __
#_ < |\| |< _#

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 19/09/2023 10:11, Computer Nerd Kev wrote:

The Natural Philosopher <tnp@invalid.invalid> wrote:

No. I am trying to establish whether one can, in essence, make a
bootable PI SD card by taking a raw card, creating the two partitions on
it and loading it up with files,

Yes. In fact if you set the OS up from scratch you don't even need
two partitions, just one FAT32 parition with the boot files
required by the Pi.

I set up a Linux system to boot off the FAT32 parition then run in
a tmpfs. So instead of an image file there's just a ZIP archive
which needs to be unpacked to the root of a FAT32-formatted SD
card. No image writer tool required, and no worries about partition
sizes.

The Pi doesn't need anything like a boot sector. The firmware is
smart enough to find the boot files on the filesystem, provided
it's FAT32.

I suppose that needs to be the first partition?
The other question why always gets me on backups is how many of the
directories in the root partition can safely be ignored?

/sys, /proc, /run would seem to be created by the machine at boot time,
and shouldn't be created as subdirs of the root partition...or should
the directories be created, but empty?

So putatively one might back up everything in / except /run, /proc &
/sys, including /boot, plonk a shiny new SD card in a host computer,
create a VFAT partition of say 256MB, and the rest as ext4, untar the
archive into the ext4 partition and move the entire contents of /boot
onto the VFAT, shove the card in a pi , apply power, and Robert should
be a relative?

The vfat partition will get mounted on the (now empty) /boot mount
point, and everything else will be where it ought to be, and /run /sys
and /proc will get generated by the boot process?

What about /dev? Is that dynamically created or not?





--
"Anyone who believes that the laws of physics are mere social
conventions is invited to try transgressing those conventions from the
windows of my apartment. (I live on the twenty-first floor.) "

Alan Sokal

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On Tue, 19 Sep 2023 08:15:03 +0100, The Natural Philosopher wrote:

On 19/09/2023 03:15, Newyana2 wrote:

"The Natural Philosopher" <tnp@invalid.invalid> wrote

| > |
| > Start menu. The raspberry icon in lower left. |
| ??? Icon? On a headless Pi Zero?
|
Ah. You might have said that in the first place. It
doesn't make much sense to ask for help and then be difficult with
people who try to help. I gather you only posted to argue.

No. I am trying to establish whether one can, in essence, make a
bootable PI SD card by taking a raw card, creating the two partitions on
it and loading it up with files, or whether there is the equivalent of a
boot sector which would mandate the use of e.g. dd to create it

Naturally all the documentation assumes a windows numpty and shows you
how to stamp the SD card with an image.

You assumed I wanted to clone the card. I don't. I want to back one up without using DD because, as the concurrent thread has identified, the
PI expands the file system to the whole card and that makes for a big
image.
I'd rather back it up as files if possible.

If at some future date I need to recreate it, instead of dd-ing the
image, I would prefer to create the two partitions and populate them
from the tarball.

Take a look at 'rsync' - its my go-to tool for backing up complete Linux
filing systems, specific partitions or lists of files and partitions to as
a named directory structure on a backup disk.

It is designed make the backup directory structure identical to the
current list of files and directories you ask it to back up, IOW, if you
rerun the sa,me command a week later it will remove any files and
directories from the previous backup session, replace any files and
directories that have been changed and add any new files and directories.

Consequently, subsequent runs will usually be much faster than the initial backup.

Want to keep older backups? Just use a cycle of backup files. SD-cards or USB-connected disks: I back up entire Linux file systems to a cycle of two
1TB WD Essentials portable disks, and do the backup immediately before the weekly Linux software update.One 1TB disk is enough to hold complete
backup cycles for both my house server this laptop and one or more RPi
backups.

Retrieving whole partition contents, directories and/or files from these backups is simple: simply mount the backup volume and copy anything you've
lost back to where it should be on your computer's filing system.


rsync manuals and software are in included in most mainstream Linux
distros.

HTH

--

Martin | martin at
Gregorie | gregorie dot org

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On Tue, 19 Sep 2023 10:35:23 +0100, The Natural Philosopher wrote:

On 19/09/2023 10:11, Computer Nerd Kev wrote:

The Natural Philosopher <tnp@invalid.invalid> wrote:

No. I am trying to establish whether one can, in essence, make a
bootable PI SD card by taking a raw card, creating the two partitions
on it and loading it up with files,

Yes. In fact if you set the OS up from scratch you don't even need two
partitions, just one FAT32 parition with the boot files required by the
Pi.

I set up a Linux system to boot off the FAT32 parition then run in a
tmpfs. So instead of an image file there's just a ZIP archive which
needs to be unpacked to the root of a FAT32-formatted SD card. No image
writer tool required, and no worries about partition sizes.

The Pi doesn't need anything like a boot sector. The firmware is smart
enough to find the boot files on the filesystem, provided it's FAT32.

I suppose that needs to be the first partition?

Nope: just whatever partition the RPi installer configured to be the boot partition.

So putatively one might back up everything in / except /run, /proc &
/sys, including /boot, plonk a shiny new SD card in a host computer,
create a VFAT partition of say 256MB, and the rest as ext4, untar the
archive into the ext4 partition and move the entire contents of /boot
onto the VFAT, shove the card in a pi , apply power, and Robert should
be a relative?

Just tell dd to copy everything from the source SD card to the new one. If
the latter card is bigger,use parted to extend the new ext4 partition and
to format the new sectors.



--

Martin | martin at
Gregorie | gregorie dot org

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 19/09/2023 20:05, Martin Gregorie wrote:

On Tue, 19 Sep 2023 08:15:03 +0100, The Natural Philosopher wrote:

No. I am trying to establish whether one can, in essence, make a
bootable PI SD card by taking a raw card, creating the two partitions on
it and loading it up with files, or whether there is the equivalent of a
boot sector which would mandate the use of e.g. dd to create it

Take a look at 'rsync' - its my go-to tool for backing up complete Linux filing systems, specific partitions or lists of files and partitions to as
a named directory structure on a backup disk.

It is designed make the backup directory structure identical to the
current list of files and directories you ask it to back up, IOW, if you rerun the sa,me command a week later it will remove any files and
directories from the previous backup session, replace any files and directories that have been changed and add any new files and directories.

Consequently, subsequent runs will usually be much faster than the initial backup.

Here's the rsync options I used to backup the partitions, firstly the
FAT partition which accounts for the lower resolution of timestamps

rsync -vax --stats --modify-window=2 --delete /boot/ <boot_backup_dir>

And the main partition with the exclusions to prevent pseudo filing
systems and temporary stuff being included when backing up live systems,
some of it shouldn't be necessary but turns out is needed

rsync -vaxHAX --stats --numeric-ids --delete --delete-excluded
--exclude=/tmp/*
--exclude=/run/*
--include=/var/lock
--exclude=/var/run
--exclude=/var/swap
--exclude=/var/cache/*
--exclude=/var/tmp/*
--exclude=/var/lib/machines/*/var/cache/
--exclude=/var/lib/mlocate/*
--exclude=/var/lib/nginx/fastcgi/*
--exclude=/var/log/journal/*
--exclude=/var/log/Xorg.*
--exclude=/lost+found/*
--exclude=*/.cache
--exclude=*/__pycache__
--exclude=*/chromium/hyphen-data/
--exclude=*/.config/chromium/*Cache*/*
/ <root_backup_dir>

---druck

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

The Natural Philosopher <tnp@invalid.invalid> wrote:

/sys, /proc, /run would seem to be created by the machine at boot time,
and shouldn't be created as subdirs of the root partition...or should
the directories be created, but empty?

/sys, /proc and /dev are filesystems which are mounted at boot time. You
need the empty directories to act as mount points, because Unix mounts filesystems on top of a directory that already exists, switching out
whatever is there.

It seems that some distros have /var/run and point a symlink from /run to
it, so it doesn't need a mount point created for them. They then mount a
tmpfs at /var/run (so /var/run is an empty directory that needs to exist to
be mounted on top of)

So putatively one might back up everything in / except /run, /proc &
/sys, including /boot, plonk a shiny new SD card in a host computer,
create a VFAT partition of say 256MB, and the rest as ext4, untar the
archive into the ext4 partition and move the entire contents of /boot
onto the VFAT, shove the card in a pi , apply power, and Robert should
be a relative?

I think that will probably work.

The vfat partition will get mounted on the (now empty) /boot mount
point, and everything else will be where it ought to be, and /run /sys
and /proc will get generated by the boot process?

One thing to watch that distros' /etc/fstab often mounts partitions based
on a UUID. So /boot on one SD will have a different UUID to another, and
the mount of /boot will fail. It'll still find the kernel from /boot, but
any time later anything wants to load something from /boot it may fail
because the mount failed. You could change the device name in /etc/fstab to
be /dev/mmcblk0p1 or similar to be sure it's the same every time.
(the Pi has a single SD slot so no chance of mixing it up with another)

'blkid' will show your partition devices with their UUIDs.

What about /dev? Is that dynamically created or not?

A dynamic filesystem mounted to a static pre-existing mount point.

Theo

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

The Natural Philosopher <tnp@invalid.invalid> wrote:

On 19/09/2023 10:11, Computer Nerd Kev wrote:

The Natural Philosopher <tnp@invalid.invalid> wrote:
The Pi doesn't need anything like a boot sector. The firmware is
smart enough to find the boot files on the filesystem, provided
it's FAT32.

I suppose that needs to be the first partition?

"It is no longer necessary for the first partition to be the FAT
partition, as the MSD boot will continue to search for a FAT
partition beyond the first one."
https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#boot-sequence

The other question why always gets me on backups is how many of the directories in the root partition can safely be ignored?

/sys, /proc, /run would seem to be created by the machine at boot time,
and shouldn't be created as subdirs of the root partition...or should
the directories be created, but empty?

/sys and /proc can be created empty, because they're mountpoints.
However I don't bother excluding them from backups personally.

So putatively one might back up everything in / except /run, /proc &
/sys, including /boot, plonk a shiny new SD card in a host computer,
create a VFAT partition of say 256MB, and the rest as ext4, untar the
archive into the ext4 partition and move the entire contents of /boot
onto the VFAT, shove the card in a pi , apply power, and Robert should
be a relative?

The vfat partition will get mounted on the (now empty) /boot mount
point, and everything else will be where it ought to be, and /run /sys
and /proc will get generated by the boot process?

Yes. Mounting the FAT32 partition at /boot is specific behaviour
for RPi OS / Debian. Linux itself doesn't care if that partition
gets mounted at all - by the time the kernel is loaded its job has
been done, but Debian probably expects it there for configuration
and upgrades.

What about /dev? Is that dynamically created or not?

The device files need backing up, or else something might have to
create them with mknod. It contains more mountpoints though, so
you can look at the list displayed by "mount" and exclude those
sub-directories of /dev if you like. Again, I don't bother to do
that.

--
__ __
#_ < |\| |< _#

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 2023-09-19, Newyana2 <Newyana2@invalid.nospam> wrote:

"The Natural Philosopher" <tnp@invalid.invalid> wrote

Start menu. The raspberry icon in lower left.

??? Icon? On a headless Pi Zero?

Ah. You might have said that in the first place.
It doesn't make much sense to ask for help and then
be difficult with people who try to help. I gather
you only posted to argue.

This is totally off topic, but I'm curious: did you
once post under the name "Mayayana"? If so, I'd like
to thank you for the quote in my .sig below.

--
/~\ Charlie Gibbs | They don't understand Microsoft
\ / <cgibbs@kltpzyxm.invalid> | has stolen their car and parked
X I'm really at ac.dekanfrus | a taxi in their driveway.
/ \ if you read it the right way. | -- Mayayana

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 20/09/2023 18:23, Charlie Gibbs wrote:

On 2023-09-19, Newyana2 <Newyana2@invalid.nospam> wrote:

"The Natural Philosopher" <tnp@invalid.invalid> wrote

Start menu. The raspberry icon in lower left.

??? Icon? On a headless Pi Zero?

Ah. You might have said that in the first place.
It doesn't make much sense to ask for help and then
be difficult with people who try to help. I gather
you only posted to argue.

This is totally off topic, but I'm curious: did you
once post under the name "Mayayana"? If so, I'd like
to thank you for the quote in my .sig below.

Er no. I don't believe I ever did. Ive been using this one a LONG time
now as my 'I'm not at work presenting my company' one for a long time.


--
"Women actually are capable of being far more than the feminists will
let them."

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

The Natural Philosopher <tnp@invalid.invalid> writes:

The other question why always gets me on backups is how many of the directories in the root partition can safely be ignored?

If you don't want to fiddle with that, there is partclone which makes
copies of partitions, smartly enough so it doesn't copy unused sectors.

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 2023-09-19, druck <news@druck.org.uk> wrote:

Here's the rsync options I used to backup the partitions, firstly the
FAT partition which accounts for the lower resolution of timestamps

rsync -vax --stats --modify-window=2 --delete /boot/ <boot_backup_dir>

And the main partition with the exclusions to prevent pseudo filing
systems and temporary stuff being included when backing up live systems,
some of it shouldn't be necessary but turns out is needed

rsync -vaxHAX --stats --numeric-ids --delete --delete-excluded --exclude=/tmp/*
--exclude=/run/*
--include=/var/lock
--exclude=/var/run
--exclude=/var/swap
--exclude=/var/cache/*
--exclude=/var/tmp/*
--exclude=/var/lib/machines/*/var/cache/
--exclude=/var/lib/mlocate/*
--exclude=/var/lib/nginx/fastcgi/*
--exclude=/var/log/journal/*
--exclude=/var/log/Xorg.*
--exclude=/lost+found/*
--exclude=*/.cache
--exclude=*/__pycache__
--exclude=*/chromium/hyphen-data/
--exclude=*/.config/chromium/*Cache*/*
/ <root_backup_dir>

I use very similar options. I add -u and I add the --delete-after
option. Oh and --numeric-ids . I've been bitten by backing up to a
system where some of the system uids (<1000) were different and on backup
got mangled. At least with numeric-only they go back as they were.

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)

On 21/09/2023 21:58, Jim Jackson wrote:

On 2023-09-19, druck <news@druck.org.uk> wrote:

Here's the rsync options I used to backup the partitions, firstly the
FAT partition which accounts for the lower resolution of timestamps

rsync -vax --stats --modify-window=2 --delete /boot/ <boot_backup_dir>

And the main partition with the exclusions to prevent pseudo filing
systems and temporary stuff being included when backing up live systems,
some of it shouldn't be necessary but turns out is needed

rsync -vaxHAX --stats --numeric-ids --delete --delete-excluded
--exclude=/tmp/*
--exclude=/run/*
--include=/var/lock
--exclude=/var/run
--exclude=/var/swap
--exclude=/var/cache/*
--exclude=/var/tmp/*
--exclude=/var/lib/machines/*/var/cache/
--exclude=/var/lib/mlocate/*
--exclude=/var/lib/nginx/fastcgi/*
--exclude=/var/log/journal/*
--exclude=/var/log/Xorg.*
--exclude=/lost+found/*
--exclude=*/.cache
--exclude=*/__pycache__
--exclude=*/chromium/hyphen-data/
--exclude=*/.config/chromium/*Cache*/*
/ <root_backup_dir>

I use very similar options. I add -u and I add the --delete-after
option. Oh and --numeric-ids . I've been bitten by backing up to a
system where some of the system uids (<1000) were different and on backup
got mangled. At least with numeric-only they go back as they were.

This is what I ended up using. I have to back up /boot every time as it
gets deleted by the first backup

#!/bin/bash
# Mounts the backup server, backs up the files in / excluding mounted
stuff and other dross
# then backs up the boot partition
# and unmounts the backup server


mount 192.168.0.100:/backup2 /mnt
cd /mnt/heating-controller

# And the main partition with the exclusions to prevent pseudo filing
systems and temporary stuff being included when backing up live systems,
some of it shouldn't be necessary # but turns out is needed

rsync -vaxHAX --stats --numeric-ids --delete --delete-excluded --one-file-system --exclude=/tmp/* --exclude=/run/* --include=/var/lock --exclude=/var/run --exclude=/var/swap --exclude=/var/cache/* --exclude=/var/tmp/* --exclude=/var/lib/machines/*/var/cache/ --exclude=/var/lib/mlocate/* --exclude=/var/lib/nginx/fastcgi/* --exclude=/var/log/journal/* --exclude=/var/log/Xorg.*
--exclude=/lost+found/* --exclude=*/.cache --exclude=*/__pycache__ --exclude=*/chromium/hyphen-data/ --exclude=*/.config/chromium/*Cache*/* / .

rsync -vax --stats --modify-window=2 --delete /boot/ ./boot
cd
umount /mnt

--
“I know that most men, including those at ease with problems of the greatest complexity, can seldom accept even the simplest and most
obvious truth if it be such as would oblige them to admit the falsity of conclusions which they have delighted in explaining to colleagues, which
they have proudly taught to others, and which they have woven, thread by thread, into the fabric of their lives.”

― Leo Tolstoy

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)