Download image and extract files into some directory. I had to use separate machine, so I copied files to my ftp server (192.168.1.28).
mount -o loop /tmp/CentOS-8-x86_64-1905-boot.iso /mnt/iso scp -r * admin@192.168.1.28:/data/pub/CentOS-8.0.1905-boot umount /mnt/iso
On my tftp/ftp server:
mkdir /data/tftpboot/networkboot/CentOS8-1905 cp /data/pub/CentOS-8.0.1905-boot/images/pxeboot/vmlinuz \ /data/tftpboot/networkboot/CentOS8-1905/ cp /data/pub/CentOS-8.0.1905-boot/images/pxeboot/initrd.img \ /data/tftpboot/networkboot/CentOS8-1905/
Add entry to PXE configuration (in my case /data/tftpboot/pxelinux.cfg/default):
LABEL InstallCentOS-8.0.1905 MENU LABEL Install CentOS 8 1905 kernel /networkboot/CentOS8-1905/vmlinuz APPEND initrd=/networkboot/CentOS8-1905/initrd.img \ ip=dhcp \ inst.repo=ftp://192.168.1.28/CentOS-8.0.1905-boot
Source:
https://docs.centos.org/en-US/8-docs/advanced-install/assembly_preparing-for-a-network-install/
Test iSCSI
The LUN is created on ReadyNAS (192.168.1.28) in group CentOS-PXE, target iqn.1994-11.com.netgear:nas:08303042:centos-pxe.
All commands are run from my laptop (T530), but it can be any Linux:
[root@T530 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.1.28 192.168.1.28:3260,1 iqn.1994-11.com.netgear:nas:051ada19:group1 192.168.1.28:3260,1 iqn.1994-11.com.netgear:nas:08303042:centos-pxe
We are interested in the second one:
[root@T530 ~]# iscsiadm -m node -T 'iqn.1994-11.com.netgear:nas:08303042:centos-pxe' -p 192.168.1.28 -l Logging in to [iface: default, target: iqn.1994-11.com.netgear:nas:08303042:centos-pxe, portal: 192.168.1.28,3260] (multiple) Login to [iface: default, target: iqn.1994-11.com.netgear:nas:08303042:centos-pxe, portal: 192.168.1.28,3260] successful.
Now we can list disks:
[root@T530 ~]# ls -l /dev/disk/by-path razem 0 lrwxrwxrwx. 1 root root 9 10-22 17:25 fc---lun-0 -> ../../sdd iqn.1994-11.com.netgear:nas:08303042:centos-pxe-lun-0 -> ../../sdd lrwxrwxrwx. 1 root root 9 10-22 16:29 pci-0000:00:0d.0-ata-1.0 -> ../../sda lrwxrwxrwx. 1 root root 10 10-22 16:29 pci-0000:00:0d.0-ata-1.0-part1 -> ../../sda1 lrwxrwxrwx. 1 root root 10 10-22 16:29 pci-0000:00:0d.0-ata-1.0-part2 -> ../../sda2 lrwxrwxrwx. 1 root root 9 10-22 16:29 pci-0000:00:0d.0-ata-2.0 -> ../../sr0
as you can see, the new volume is linked as /dev/sdd usualy it will be sda or sdb.
Now we can "unmount" as it won't be needed.
iscsiadm -m node -T 'iqn.1994-11.com.netgear:nas:08303042:centos-pxe' -p 192.168.1.28 -u
Configure network install
You will need: tftp, dhcp and ftp server and CentOS installation ISO file. Unpack ISO content to ftp server to some (CentOS7-1908-minimal in my case) directory.
Add entry in /data/tftpboot/pxelinux.cfg/default file:
LABEL InstallCentOS7 MENU LABEL Install CentOS 7 1908 KERNEL /networkboot/CentOS7-1908/vmlinuz APPEND initrd=/networkboot/CentOS7-1908/initrd.img inst.repo=ftp://192.168.1.28/CentOS7-1908-minimal
You have to create networkboot/CentOS7-1908 directory in your tftp root directory. In my (ReadtNAS) case /data/tftpboot/, so full path is /data/tftpboot/networkboot/CentOS7-1908.
Files vmlinux and initrd.img are from ISO from isolinux directory. Because my ftp root is /data/pub/ I typed:
cp /data/pub/CentOS-7-x86_64-Minimal-1908/isolinux/{vmlinuz,initrd.img} /data/tftpboot/networkboot/CentOS7-1908
Now you should be able to run CentOS installation from network.
Installation
Boot computer from network (PXE BOOT) choose Install CentOS 7 1908 from pxe menu.
Run standard installator. As Installation destination add disk in Specialized & Network Disks. In Add iSCSI Target... enter 192.168.1.28 as Target IP Address and click Start discovery. Choose desired node (if you have more than one you should know what do do) and click Log in and (after a while) OK.
Now you should have some drive (sda, sdb or similar) - select it and click Done. Probably there will be Error checking storage configuration message - click Full disk summary and boot loader ... and unselect your disk as a boot device by clicking Do not install boot loader. Click Done.
Now be sure the name of your network card. In my case it is enp0s3.
You can add more installation repositories, but it is not necessary now.
Click Begin installation. It will take a while - in the meantime you can set root password.
When installation is complete switch to console (Ctrl-Alt-F2) and copy vmlinuz and initramfs files to your tftp server:
scp /mnt/sysimage/boot/vmlinuz-3.10.0-1062.el7.x86_64 /mnt/sysimage/boot/initramfs-3.10.0-1062.el7.x86_64.img user@192.168.1.28:
and place this two files in tftp directory:
mv /home/user/vmlinuz-3.10.0-1062.el7.x86_64 /home/user/initramfs-3.10.0-1062.el7.x86_64.img /data/tftpboot/networkboot/CentOS7-1908
as you can see I use the same directory as in network install, but it is no necessary.
Now it is time to final PXE configuration on tftp server prepare entry similar to:
LABEL Centos7 MENU DEFAULT MENU LABEL Cent OS 7 kernel /networkboot/CentOS7-1908/vmlinuz-3.10.0-1062.el7.x86_64 root=/dev/mapper/centos-root ro netroot=iscsi:@192.168.1.28::::iqn.1994-11.com.netgear:nas:08303042:centos-pxe rd.iscsi.initiator=iqn.1994-05.com.redhat:2f3474eec735 vconsole.keymap=pl LANG=en_GB.UTF-8 console=tty0 ip=enp3s0:dhcp rhgb quiet append initrd=/networkboot/CentOS7-1908/initramfs-3.10.0-1062.el7.x86_64.img
You can check "magic numbers" on freshly installed system by:
[anaconda root@localhost /]# cat /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.1994-05.com.redhat:2f3474eec735 [anaconda root@localhost /]# iscsiadm -m session tcp: [1] 192.168.1.28:3260,1 iqn.1994-11.com.netgear:nas:08303042:centos-pxe [anaconda root@localhost /]# grep root /mnt/sysimage/etc/fstab /dev/mapper/centos-root / xfs defaults,_netdev,_netdev 0 0
Now you can click Reboot.
After some time you have network booted system.
Configure iPXE
Instead of PXE I should be possible to use iPXE. For now I can only boot iPXE - then I return to PXE because it works.
Download binary (on ReadyNAS - TFTP server) and move to tftp directory:
cd /tmp/ wget http://boot.ipxe.org/undionly.kpxe mv /tmp/undionly.kpxe /data/tftpboot/
Not to have to flash network card ROM I use PXE chainloading. Change (on Gargoyle - DHCP server):
dhcp-match=set:ipxe,175 dhcp-boot=tag:!ipxe,undionly.kpxe,,192.168.1.28 dhcp-boot=menu.ipxe,,192.168.1.28
It should be more convenient way to upgrade kernel in future.
For now, in menu.ipxe I've entry:
:pxelinux set 210:string tftp://192.168.1.28/ chain ${210:string}pxelinux.0 || goto failed goto start
Sources:
Diskless iSCSI boot with PXE HOWTO
How to chainload ipxe using dnsmasq?
Centos 7 Diskless Environment – PXE booting OpenStack compute nodes from iSCSI SAN
Last time I've prepared Gargoyle as dhcp and tftp server.
Prepare tftp on ReadyNAS
To enable tftp on ReadyNAS I've installed Tftp Server for Readynas. All files have to be in tftpboot share, so I've created it.
I have to copy all syslinux files (can be copied from gargoyle) to this share, so it contains (full path):
/data/tftpboot/pxelinux.cfg /data/tftpboot/pxelinux.cfg/default /data/tftpboot/ldlinux.c32 /data/tftpboot/libutil.c32 /data/tftpboot/menu.c32 /data/tftpboot/pxelinux.0
Reconfigure DHCP
Now I have to change dhcp server configuration. On gargoyle, in file /etc/dnsmasq.conf:
dhcp-boot=pxelinux.0,,192.168.1.28
as you can see, now it has address of my ReadyNAS (192.168.1.28).
You can comment out lines
enable-tftp tftp-root=/var/lib/misc/tftpboot
tftp on gargoyle is not need any more.
Now you should be able to boot any PC using PXE from ReadyNAS.
Boot to CentOS 7 (or any Linux) Installer
Now you can try to run Cent OS installer from network. You have to prepare FTP share and copy there files from installation image (CD/DVD). I've copied it to ReadyNAS to pub/CentOS7-1708 share .
If you use proftpd you have to create /etc/frontview/proftpd/proftpd.conf.overrides file to allow anonymous access:
UserAlias anonymous guest <Anonymous /var/ftp/pub> User guest Group guest <Directory *> <Limit WRITE> DenyAll </Limit> </Directory> <Directory incoming> <Limit READ > DenyAll </Limit> <Limit STOR> AllowAll </Limit> </Directory> </Anonymous>
Create /data/tftpboot/networkboot/CentOS7-1708 directory and copy there files:
/data/tftpboot/networkboot/CentOS7-1708/initrd.img /data/tftpboot/networkboot/CentOS7-1708/vmlinuz
In /data/tftpboot/pxelinux.cfg/default file add lines:
LABEL InstallCentOS7 MENU DEFAULT MENU LABEL Install CentOS 7 KERNEL /networkboot/CentOS7-1708/vmlinuz APPEND initrd=/networkboot/CentOS7-1708/initrd.img inst.repo=ftp://192.168.1.28/CentOS7-1708-minimal
notice that ftp path is relative to pub share.
Now you should be able to boot to Cent OS installer.
Prepare NFS share
I've done it on ReadtNAS. You have to set in Settings - Network Access - NFS - Advanced settings: No Root Squash.
Be careful because you have to provide full path when mount share. My share is pub/root, so I have to mount as /data/pub/root:
mount -vt nfs 192.168.1.18:/data/pub/root /mnt/nfsroot/
As you can see, I've mounted this share in /mnt/nfsroot directory.
Install Cent OS
yum install --releasever=7 --installroot=/mnt/nfsroot \@base \@core
when You want to install another packages you can see error:
Wczytane wtyczki: fastestmirror, langpacks Loading mirror speeds from cached hostfile There are no enabled repos. Run "yum repolist all" to see the repos you have. To enable Red Hat Subscription Management repositories: subscription-manager repos --enable <repo> To enable custom repositories: yum-config-manager --enable <repo>
you will have to
cp /etc/yum.repos.d/CentOS-Base.repo /mnt/nfsroot/etc/yum.repos.d/
Now you can add some GUI packages:
yum install --releasever=7 --installroot=/mnt/nfsroot \@fonts \@gnome-desktop \@x11
Based on: CentOS 7: Install PXE Boot server for thin client with NFS.
On CentOS 7 you can log all commands to syslog an then to local file or even to remote server.
Send all commands to syslog
Create file /etc/sysconfig/bash-prompt-xterm:
RETRN_VAL=$?;logger -p local6.debug "$(whoami) [$$]: $(history 1 | sed "s/^[ ]*[0-9]\+[ ]*//" ) [$RETRN_VAL]"
and change, to be executable:
chmod a+x /etc/sysconfig/bash-prompt-xterm
Configure syslog to send messages from local6 facility to separate file
Create file /etc/rsyslog.d/bash.conf:
local6.* /var/log/commands.log
finally:
service restart rsyslog
Now you can monitor commands:
tail -f /var/log/commands.log
Log command using audit
Alternatively you can use audit - create /etc/audit/rules.d/bash_history.rules:
-a exit,always -F arch=b64 -S execve -a exit,always -F arch=b32 -S execve
but logs are not very human friendly:
grep EXECVE /var/log/audit/audit.log
and you may also want to log execvp, execl, execveat etc.
Sources:
https://askubuntu.com/questions/93566/how-to-log-all-bash-commands-by-all-users-on-a-server
https://unix.stackexchange.com/questions/86000/how-can-you-log-every-command-typed
http://whmcr.com/2011/10/14/auditd-logging-all-commands/
Setting up PXE booting server.
Set up DHCP server:
On Gargoyle (my dhcp server) router I've add to /etc/dnsmasq.conf:
dhcp-boot=pxelinux.0 enable-tftp tftp-root=/var/lib/misc/tftpboot
of course you can try to use uci instead, if you know how - I couldn't.
Set up files
Download newest syslinux from Linux kernel site (in my case: syslinux 6.03).
Copy
- bios/com32/libutil/libutil.c32
- bios/com32/elflink/ldlinux/ldlinux.c32
- bios/com32/menu/menu.c32
- bios/core/pxelinux.0
to /var/lib/misc/tftpboot directory.
Create pxelinux.cfg directory and default file inside:
mkdir /var/lib/misc/tftpboot/pxelinux.cfg/ vi /var/lib/misc/tftpboot/pxelinux.cfg/default
The default file should have content like this:
DEFAULT menu.c32 TIMEOUT 300 ALLOWOPTIONS 0 PROMPT 0 MENU TITLE My new PXE Boot Menu LABEL BootLocal MENU LABEL ^Boot Local (HDD) LOCALBOOT 0
now you should be able to test booting from any computer in local network.
Of course this is dummy example as it can only boot from local disk.
As you can see in logs (using logread command on Gargoule router):
Fri Sep 27 19:25:47 2019 daemon.info dnsmasq-tftp[20876]: sent /var/lib/misc/tftpboot/pxelinux.0 to 192.168.1.163 Fri Sep 27 19:25:47 2019 daemon.info dnsmasq-tftp[20876]: sent /var/lib/misc/tftpboot/ldlinux.c32 to 192.168.1.163 Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/44454c4c-3400-1037-8034-b1c04f30354a not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/01-84-2b-2b-bd-47-59 not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C0A801A3 not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C0A801A not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C0A801 not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C0A80 not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C0A8 not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C0A not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C0 not found Fri Sep 27 19:25:47 2019 daemon.err dnsmasq-tftp[20876]: file /var/lib/misc/tftpboot/pxelinux.cfg/C not found Fri Sep 27 19:25:47 2019 daemon.info dnsmasq-tftp[20876]: sent /var/lib/misc/tftpboot/pxelinux.cfg/default to 192.168.1.163
My computer tried to find following configs files:
- his UUID (44454c4c-3400-1037-8034-b1c04f30354a)
- his MAC address (01-84-2b-2b-bd-47-59) with 01 prefix
- full IP address (C0A801A3) - C0.A8.01.A3 is equal to 192.168.1.163.
- 7 larger subnets (by 4 bits) - C0A801A to C.
- finally default file
You can use it later to per host configurations.
To backup running QEMU/KVM machine I use script made by Daniel Berteaud. I've installed it in this way:
cd /usr/local/sbin/ wget "http://gitweb.firewall-services.com/?p=virt-backup;a=blob_plain;f=virt-backup;hb=HEAD" -O virt-backup-0.2.17-1.pl ln -s virt-backup-0.2.17-1.pl virt-backup chmod u+x virt-backup-0.2.17-1.pl
as you can see current version is 0.2.17-1
There is also a fork of virt-backup: github.com/vazhnov/virt-backup.pl - it's based on some earlier version.
Example command to backup centos-test machine (name shown by virsh list) to /mnt/backups directory.
virt-backup --action=dump --no-snapshot --compress --shutdown --shutdown-timeout=300 --vm=centos-test --backupdir=/mnt/backups
If you forgot to set-up power policy during container creation you can easy modify it:
docker update --restart=always my-container
you can check it:
docker@docker:~$ docker inspect my-container | grep -A3 "RestartPolicy" "RestartPolicy": { "Name": "always", "MaximumRetryCount": 0 },
If your proxy server (squid and/or dansguardian) is not on your gateway you can also set it up to be transparent.
In my examples gateway ha address 192.168.1.1, and squid server has address 192.168.1.28 and listen on 8080 port.
There is configuration using iptables directly:
iptables -t nat -I PREROUTING -i eth0 -s ! 192.168.1.28 -p tcp --dport 80 -j DNAT --to 192.168.1.28:8880 iptables -t nat -I POSTROUTING -o eth0 -s 192.168.1.0/24 -d 192.168.1.28 -j SNAT --to 192.168.1.1 iptables -I FORWARD -s 192.168.1.0/24 -d 168.13.28 -i eth0 -o eth0 -p tcp --dport 8880 -j ACCEPT
and this is the same in Gargoyle /etc/config/firewall file (you can edit it or use uci add firewall commands):
config redirect option name 'P12 to Squid DNAT' option src 'lan' option proto 'tcp' option dest_port '8080' option src_dport '80' option src_dip '! 192.168.1.1' option dest_ip '192.168.1.28' option src_ip '! 192.168.1.28' config redirect option name 'P12 to Squid SNAT' option dest 'lan' option proto 'tcp' option src_dip '192.168.1.1' option dest_ip '192.168.1.28' option src_ip '192.168.1.0/24' option target 'SNAT' config rule option name 'P12 to Squid' option dest 'lan' option dest_port '8080' option proto 'tcp' option src_ip '192.168.1.0/24' option dest_ip '192.168.1.28' option target 'ACCEPT'
after editing /etc/config/firewall file you have to restart firewall:
/etc/init.d/firewall restart
Today I wrote my firs application for ReadyNAS OS. It's very, very simple. It just for Proxy Auto Configuration (PAC) / Web Proxy Autodiscovery Protocol (WPAD) so I called it wpad ;-)
The whole "application" serve only one file wpad.dat. It fits in two files: /apps/wpad/http.conf and /apps/wpad/wpad.dat itself. The content of /apps/wpad/http.conf is:
<VirtualHost *:80> ServerAdmin admin@localhost ServerName wpad DocumentRoot /apps/wpad ErrorLog /apps/wpad/error.log LogLevel warn </VirtualHost>
Now when apache starts it creates link:
root@NAS:~# ll /etc/apache2/sites-enabled/090-wpad.conf lrwxrwxrwx 1 root root 20 Mar 22 21:15 /etc/apache2/sites-enabled/090-wpad.conf -> /apps/wpad/http.conf
To more detailed info about applications for ReadyNAS OS see ReadyNAS Applications Specification.
To have WPAD worked I had to configure also my router:
To resolve wpad as 192.168.1.12 (this is address of my NAS) in /etc/dnsmasq.conf I added:
address=/wpad/192.168.1.12
and to in /etc/config/dhcp file in section config dhcp 'lan':
list dhcp_option '252,http://wpad/wpad.dat'
Installation
yum install https://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm yum install puppetserver
add following lines to /etc/puppetlabs/puppet/puppet.conf, of course, change puppetmaster to your host name:
dns_alt_names = puppet,puppetmaster [agent] runinterval = 1m
you can reduce memory usage - edit /etc/sysconfig/puppetserver file and change JAVA_ARGS to:
JAVA_ARGS="-Xms256m -Xmx384m -XX:MaxPermSize=256m"
then run service:
systemctl enable puppetserver systemctl start puppetserver
First manifests
To communicate with agents puppet use 8140 port - we try to open it using puppet in three steps:
- Install module to manage firewalld
- Create new puppet service
- Enable this service for public zone
Module installation
First we need module to manage firewalld - to install it create manifest file, ie. puppet.test.firewalld-install.pp (as you can see I used /root directory, but it does'n matter):
$module_firewalld = 'crayfishx-firewalld' exec { 'install-puppet-module-firewalld': command => "puppet module install ${module_firewalld}", unless => "puppet module list | grep ${module_firewalld}", path => ['/bin', '/opt/puppetlabs/bin'] }
and test (--noop) first manifest:
[root@puppetmaster ~]# puppet apply puppet.test.firewalld-install.pp --noop Notice: Compiled catalog for puppetmaster.lan in environment production in 0.18 seconds Notice: /Stage[main]/Main/Exec[install-puppet-module-firewalld]/returns: current_value notrun, should be 0 (noop) Notice: Class[Main]: Would have triggered 'refresh' from 1 events Notice: Stage[main]: Would have triggered 'refresh' from 1 events Notice: Applied catalog in 1.45 seconds
after this you can apply (without --noop):
[root@puppetmaster ~]# puppet apply puppet.test.firewalld-install.pp Notice: Compiled catalog for puppetmaster.lan in environment production in 0.18 seconds Notice: /Stage[main]/Main/Exec[install-puppet-module-firewalld]/returns: executed successfully Notice: Applied catalog in 14.33 seconds
of course you could just run:
puppet module install crayfishx-firewalld
but it would be too simple ;-)
Create service
Now we can create firewalld service. Create file puppet.test.firewalld-service.pp:
firewalld::custom_service{'puppet': short => 'puppet', description => 'Puppet Client access Puppet Server', port => [ { 'port' => '8140', 'protocol' => 'tcp', }, { 'port' => '8140', 'protocol' => 'udp', }, ], }
and apply this:
[root@puppetmaster ~]# puppet apply puppet.test.firewalld-service.pp Notice: Compiled catalog for puppetmaster.lan in environment production in 0.27 seconds Notice: /Stage[main]/Main/Firewalld::Custom_service[puppet]/File[/etc/firewalld/services/puppet.xml]/ensure: defined content as '{md5}3fc4d356e7cb57739c8ceb8a0b483eaa' Notice: /Stage[main]/Main/Firewalld::Custom_service[puppet]/Exec[firewalld::custom_service::reload-puppet]: Triggered 'refresh' from 1 events Notice: Applied catalog in 1.32 seconds
(I cut warnings about depreciated validate functions)
As you can see there is a new file:
[root@puppetmaster ~]# more /etc/firewalld/services/puppet.xml <?xml version="1.0" encoding="utf-8"?> <service> <short>puppet</short> <description>Puppet Client access Puppet Server</description> <port protocol="tcp" port="8140" /> <port protocol="udp" port="8140" /> </service>
Again we could create this file manually, but what for?
Enable service
It's time to use this service. Create file puppet.test.firewalld-apply.pp:
firewalld_service { 'Allow puppet from the public zone': ensure => 'present', service => 'puppet', # zone => 'external', }
(because public id default parameter zone can be omitted) and apply this:
[root@puppetmaster ~]# puppet apply puppet.test.firewalld-apply.pp Notice: Compiled catalog for puppetmaster.lan in environment production in 0.15 seconds Notice: /Stage[main]/Main/Firewalld_service[Allow puppet from the public zone]/ensure: created Notice: Applied catalog in 2.04 seconds
and check:
[root@puppetmaster ~]# firewall-cmd --list-services ssh dhcpv6-client dhcp dns puppet
This way is more interesting than boring:
firewall-cmd --add-service=puppet firewall-cmd --permanent --add-service=puppet firewall-cmd --reload
Summary
As you can see setting up a test puppet master (and use them) is not so difficult. Configuration files and manifests are easy to understand.
Next step is to manage agents on other hosts...
Add comment