Backing Up VM Files with BORG

BorgBackup, or Borg, is a de-duplicating open source program that also offers compression and encryption. The main goal of Borg is to provide efficient and secure storage of backed up data. In this article, the author shows how Borg can securely back up a VM, and has added some of his own script modifications to make it more efficient.

I have been trying to back up VMs with various backup tools. This turned out to be expensive for the volume of data that seems to be growing larger every day. It was becoming important for me to back up data and at the same time reduce the cost of storage, which is calculated per GB. If the data is to be stored as per HIPPA regulations, the costs get even higher. Backing up VM files was a challenge, particularly when done live or in real-time.

As a consultant, I had to deliver a cost-effective tool to back up data and store it in an encrypted format.

The client was advised to use KVM on CentOS instead of VMware and Windows. This first option proved successful and the performance was better than the VMware-Windows combination. It was necessary to use a backup tool to store the data in a safe place even when at rest, as it contained PMI data. I scoured Google for all the possible tools prior to deciding on one — and the options were either beyond budget or did not deliver what was needed, i.e., space, efficient storage, speed, data encryption compression, robustness, being time tested, reliable, and easy to use.

I eventually landed at the Borg site (https://borgbackup.readthedocs.io), but I was sceptical about it as no one had mentioned it, nor had I heard of it earlier. But I read through the site, found it very interesting, and decided to try out Borg.

It turned out to perfectly match my needs, as it had:

Space efficient storage

Speed

Data encryption

Compression

Off-site backups

Backups mountable as file systems

And it was free and open source software.

Borg is rather easy to install on multiple platforms and offers single-file binaries that do not require anything to be installed — you can just run them on the following platforms:

Linux

Mac OS X

FreeBSD

OpenBSD and NetBSD (no xattrs/ACLs support or binaries yet)

Cygwin (experimental, no binaries yet)

Linux sub-system of Windows 10 (experimental)

Installation

Borg x86/x64 AMD/Intel compatible binaries (generated with PyInstaller) are available on the release pages for Linux: glibc >= 2.13 (okay for most supported Linux releases). Older glibc releases are untested and may not work.

To install the binary, just drop it into a directory in your PATH, make Borg readable and executable for its users and then you can run it, as follows:

sudo cp borg-linux64 /usr/local/bin/borg

sudo chown root:root /usr/local/bin/borg

sudo chmod 755 /usr/local/bin/borg

Dependencies

To install Borg from a source package (including PIP), I had to install the following dependencies first:

Python 3 >= 3.4.0, plus development headers. Even though Python 3 is not the default Python version on most systems, it is usually available as an optional install.

OpenSSL >= 1.0.0, plus development headers.

libacl and libattr (as libacl depends on libattr), plus development headers.

It has bundled code of the following packages, but Borg, by default (see setup.py if you want to change that) prefers a shared library if it can be found on the system (lib + dev headers) at build time:

liblz4 >= 1.7.0 (r129)

libzstd >= 1.3.0

libb2

Some Python dependencies and Pip automatically installed themselves for me.

I followed the steps indicated at the Borg official site:

https://borgbackup.readthedocs.io/en/stable/quickstart.html.

A step-by-step example is given below:
1. #!/bin/bash
2. set -euo pipefail
3. IFS=$’\n\t’
4. #Written By Arvind GK
5. #Part 2 : To backup Second VM Disk
6. This File name backup-vm-1st-disk.sh
7. DOMAIN=”$1”
8. UUID=$(sudo virsh dumpxml $DOMAIN | grep uuid | cut -d”>” -f2 | cut -d”<” -f1)
9.
10. DISK_LOCATION=$(sudo virsh domblklist “$DOMAIN” | grep hda | tr -s “ “ | cut -d” “ -f2-)
11.
12. DISK_LOCATION1=$(sudo virsh domblklist “$DOMAIN” | grep hdb| tr -s “ “ | cut -d” “ -f2-)
13.
14. # Export Borg encryption key , You can also modify this to using public and private key. Here I have user simple passphrase as example.
15. export BORG_PASSPHRASE=”1234”
16.
17. # Export the rate-limited remote shell
18. #export BORG_RSH=”/usr/local/bin/pv-wrapper ssh”
19.
20. Create the snapshot to a temp file on the Linux drive
21.
22. sudo virsh snapshot-create-as --domain $DOMAIN \
23.
24. tempsnap “Temporary snapshot used while backing up $DOMAIN” \ --disk-only --diskspec $DISK_LOCATION,file=”/img/$DOMAIN-tempsnap.qcow2” \ --atomic
25.
26. Commit the changes that took place in the Windows drive while
27. Sync the virsh metadata with reality by deleting the metadata
28. external image right now but that is not implemented yet)
29.
30. sudo virsh snapshot-delete “$DOMAIN” tempsnap --metadata
31.
32. (they have already been merged back into the original image) sudo rm -f “/img/$DOMAIN-tempsnap.qcow2” sudo rm -f “$DISK_LOCATION1”
33.
34. fi
35. }
36.
37. Stupid hack: make a list of all the files we *don’t* want to save so we can exclude them from the backup
38.
39. find $(dirname “$DISK_LOCATION”) ! -type d ! -wholename “$DISK_LOCATION” | awk ‘{print “sh:” $0;}’ > iso-exclusions
40. the --read-special flag tells borg to follow the symlink & read the block device directly
41.
42. Remove the temp file for the exclusions rm iso-exclusions || /bin/true
43. if it already succeeded and this ensures all guest disk access is on
44. Prune the backups on the server that are older than a certain date echo “Pruning old backups”
45. borg prune -v --list /backup --prefix “$DOMAIN-” --keep-within=1m & wait || /bin/true
46. echo “Done”

I modified this script by adding Borg to compress, encrypt and backup.

I used BLAKE2b for encrypting, as it had no legal implications on licensing and further, no history of anyone breaking the encryption.

Initialising the directory

Using Borg with the local repository, repokey encryption and BLAKE2b is often faster, since Borg 1.1 (to know more about why it is faster, refer to the document at https://blake2.net/blake2.pdf )

$ borg init --encryption=repokey-blake2 /path/to/repo

In the above example I used key encryption ‘repokey-blake2’ to encrypt and the repository path ‘/path/to/repo’ to store the backedup files; you can choose any path that has enough space to backup your files.

Here is an example of how to take a backup of the VM:

/bin/sh /qnap/backup-scripts/rcm-backup-vm-1st-disk.sh Windows2012-VM1

An output sample is given below:

[root@se_vm2 ~]# /backup/backup-scripts/rcm-backup-vm-1st-disk.sh Window2012-VM1 /home/admin/VM-Storage/vm1-c-100gb.qcow2

/home/admin/VM-Storage/vm1-d-150gb.qcow2

Domain snapshot tempsnap created

/home/admin/VM-Storage/vm1-d-150gb.tempsnap

Saving backup as Window2012-VM1-2018-07-11

Archive Window2012-VM1-c-2018-07-11 already exists

 

Block commit: [100 %]

Successfully pivoted

Block commit: [100 %]

Successfully pivoted

Domain snapshot tempsnap deleted

Pruning old backups

Keeping archive: Window2012-VM1-c-2018-07-11 Wed, 2018-07-11 20:43:10

[1068528431bd4a2c 444654f9fba25d51da35 123aee0c71bc062975f76dcf5770]

Done

Target Source

------------------------------------------------

hda /home/admin/VM-Storage/vm1-c-100gb.qcow2

hdb /home/admin/VM-Storage/vm1-d-150gb.qcow2

When you see the logs, you will notice that Borg deletes old backed up files after the specified days.

To prune old backups, the code is as follows:

borg prune -v --list /backup --prefix “$DOMAIN-” --keep-within=1m & wait || /bin/true

The logs will look something like what follows:

Keeping archive: Window2012-VM1-d-2018-07-12 Thu, 2018-07-12 12:40:33

[3601dbf1a489da61 dc05d61d2387c2693 6922e97100c896d0c5f2 42b0fb59352]

Keeping archive: Window2012-VM1-c-2018-07-12 Thu, 2018-07-12 04:00:03

[005c1d93647328f09 37a73363857bd1f151deb942d4c7159a8 d815885620e01f]

Keeping archive: Window2012-VM1-d-2018-07-11 Wed, 2018-07-11 21:40:27

[947c1c62b6163a12 63bab9c6bff1a75ff5b2b86007a521 e1683653e77d285fb3]

Keeping archive: Window2012-VM1-c-2018-07-11 Wed, 2018-07-11 20:43:10

[1068528431bd4a 2c444654f9fba25d51da35123aee0c71bc 062975f76dcf5770]

Done

 

Extracting file from archive

 

# Extract entire archive

$ borg extract /path/to/repo::my-files

 

# Extract entire archive and list files while processing

$ borg extract --list /path/to/repo::my-files

 

# Verify whether an archive could be successfully extracted, but do not write files to disk

$ borg extract --dry-run /path/to/repo::my-files

 

# Extract the "src" directory

$ borg extract /path/to/repo::my-files home/USERNAME/src

 

# Extract the "src" directory but exclude object files

$ borg extract /path/to/repo::my-files home/USERNAME/src --exclude '*.o'

 

# Restore a raw device (must not be active/in use/mounted at that time)

$ borg extract --stdout /path/to/repo::my-sdx | dd of=/dev/sdx bs=10M

Note: For extraction, you will need a passphrase, as you will be asked for it.

Enable GingerCannot connect to Ginger Check your internet connection
or reload the browserDisable in this text fieldEditEdit in GingerEdit in Ginger×Enable GingerCannot connect to Ginger Check your internet connection
or reload the browserDisable in this text fieldEditEdit in GingerEdit in Ginger×

Read More

Bharini Nakshatras

 Outrageous' is the single word which summarizes Bharani's fundamental nature and functioning.It swings between total inverses like obliviousness and intelligence, development and immaturity,life and passing. Bharani speaks to the longing of life. Much the same as a kid needs to encounter the entirety of its environmental factors, Bharani locals want to encounter their environmental factors without limit. 

They may pass by intuition as opposed to reason. A large portion of their sentiments and wants are overpowering to such an extent that almost no should be possible to limit them. Without appropriate direction and comprehension, Bharani locals are prepared to hop into everything without exception without alert or thought. 

Bharani locals for the most part have enormous, expressive eyes, an unmistakable head, medium measured lips. Their grin is their most discernable element. Head and Bottom Parts of the Feet are the body parts identified with this nakshatra. It is extremely simple for Bharani locals to move starting with one activity then onto the next without thinking about the longterm sway. Bharani quite often attempts to break the normal practices or restrictions one way or the other. 

Whole Bharini Nakshatra falls in Sign Aries whose ruler is Mars and the leader of this Nakshatra is Venus. Mars and Venus are the two planets generally worried about ripeness and interchange with the sexes.Nowadays,it has gotten popular to connect men with Mars and ladies with Venus, yet actually the male makes the sperm (Venus), while it is ladies who make and house (Mars) the new conceived. Combination of Mars and Venus conveys an energy like that of Bharani. 

Saturn arrives at its most extreme weakness in this nakshatra.Bharani is the warm spot of innovativeness and richness, while Saturn is a cool planet of check and postponement. It is anything but difficult to perceive how they don't go together. Different planets typically well in Bharani, as long as Venus has a refined position in the outline. 

Callings :All Bharani locals have an imaginative inclination inside them.Babysitters, Nannies, Nursery School Teachers; All callings including children.Volcanic and Earthquake Experts; Geophysicists; Biologists and Microbiologists; Cooks, Caterers, Hoteliers,Seed and Fertilizer Industry. Engine Sports;Film and Entertainment Industry; Photographers; Models; All callings including the utilization of sex and marvelousness. 

first Pada :Planets here can be very self-driven and can regularly irritate others without significance to. this pada has inharent determination and drive to accomplish aim.Sun, Mars and Jupiter are particularly solid in this pada. 

second Pada :The accentuation here is on administration and hardwork. Planets here capacity in a selfless manner. A specific level of association is seen even in normally outrageous and clamorous Bharani mode. Mercury, Mars and Rahu are more qualified for giving great outcomes here. 

third Pada : There is anyway no feeling of any cutoff points here with regards to sex and connections, which can be either something worth being thankful for or an awful thing relying on the developmental degree of a person.Being a pushkara navamsa pada it considers satisfaction of one's cravings. Mercury, Venus and Rahu are most appropriate for using the energies of this pada. 

fourth Pada : It can be a profoundly imaginative and unique pada if its touchy energy can be directed appropriately. Sun, Mars and Ketu are solid here however may not generally give useful outcomes. Just the common benefics Jupiter and Venus are appropriate for using the energies of this pada in a shrewd, non reckless way. 

Bharini individuals are generally solid to conquer numerous disturbance in life.They groan and cry like newborn children in some cases, yet nothing holds them down for a really long time. Bharini individuals go through enormous changes and revolutionary changes for the duration of their lives. This is a result of the extraordinary idea of Bharini Nakshatra. 

Bharani types are generally eager and vigorous in their way to deal with life. They have a solid feeling of experience in the fields they seek after. It is seen that they can possibly accomplish their objectives in the event that they are under steady direction from a more develop and more shrewd source. 

Model: Bharini is one of the most sexual nakshatras of the zodiac. Sigmund Freud, the popular psychoanalyst, had noticeable Bharani positions and no big surprise then he decreased each action of this world to sexuality. 

Anyway the more developed Bharani types understand that this power of fascination has more to do with affection, concordance and cognizant solidarity. Sexuality is only one of its numerous expressions.Imelda Marcos, perhaps the most extravagant lady in Asia, has her Moon put in Bharani. Bharani's propensity to give material limits is intensified in her life. 

She turned out to be notable for her enormous shoe collection,whose number ran into thousands.She additionally had brutal change in life which is another characteristic of Bharini.Diva of Pop Medonna, Monica Lewinsky their venus is in Bharini and we as a whole ability they came in lime light utilizing their sex bid.

Read More

Mrigasira Nakshatra

PERSONALITY

Shakti (Power): Prinana Shakti (The Power to “Give Fulfilment and Joy”)

Symbol: Deer’s / Antelope’s Head/ Arched Bow

Deity: Soma, Chandra (The Moon God), God of Soma (Nectar) or Immortality

Favourable: Light Hearted Actions, Travel, Exploration, Sexual Activity, Arts, Healing and Rejuvenation, Education, Socializing, Shifting Home, Moving to a New Location, Communication, Religion and Spirituality, Sales and Advertisement, Taking New Name, Art and Creative Pursuits, Socializing and Making New Friends, Health and Revitalization, Gardening, Advertising, Spiritual initiations

Unfavourable: Marriage, Harsh or Aggressive Action such as Confrontation or Combat, Pre-nuptial Ceremonies, Long Term Decisions.

Key Themes

Spirituality, Meditation and Substance Abuse

Pleasure-seeking and Multiple Relationships

Illegitimate Children, Perfumes and Brewing

Key Points

Deer is a lunar animal, swift, lightness, fickleness

Initially marriage problems. Under suspicions nature and little quarrels.

Bhudda was also born in this Nakshatra.

Birth sign of skhakthi, Prarvati & World of manifestation on this Nakshatra.

Curiosity, Hunting, Searching connected to Nakshatra.

More

Star: Mrigashira

Rasi (Zodiac): Taurus / Gemini

Range: 23o 20’ Taurus – 06o 40’ Gemini

Padas: Leo, Virgo, Libra, Scorpio

Ruling Planet: Mars

Meaning: “The Deer’s Head” or “The Benevolent”

Indication: “The Searching Star”

Guna (Quality): Tamas

Body Part: Eyes and Eyebrows

Gana (Race): Deva (Divine, God-Like Dispositions)

Purushartha (Goal): Moksha (Spiritual Liberation/ Emancipation)

Tridosha: Pitta (Bile or Fire + Water)

Nature/ Category of Star: Mridu (Soft/ Mild/ Tender)

Varna (Caste): Farmer/ Servant

Gotra (Clan): Pulastya (name translates as “One having Smooth Hair”)

Direction: South

Pancha Mahabhuta (Element): Prithvi (Earth)

Basis Above: To Extend

Basis Below: To Weave or Produce Clothing

Desire: To Gain Lordship over Plants

Result of Shakti, Basis, & Desire: Make the World Enjoyable

Activity: Passive

TriMurthi (Process): Vishnu (Maintain)

Direction of Mouth / Motion: Tiryanga Mukha (Looking Straight/ Looking Forward/ Level)

Yoni (Gender): Female

Animal Symbol: Serpent

Bird: Hen, Garuda/ Eagle, Pigeon

Sounds: Ve, Vo, Kaa, Kee

Tree/ Plant: Khair, Khadira, Milmesha/Cutch Tree (Latin Name: Acacia Catechu)

Colour: Silver Grey

Elements connected to Mrigasira Nakshatra

Deity	Soma
Symbol	Deer’s Head
Planet	Mars
Purushartha	Moksha
Gana	Deva
Varna	Farmer/ Servant
Element	Prithvi (Earth)
TriMurthi	Vishnu (Preserver)
Animal	Female Serpent
Bird	Hen)
Tree	Cutch Tree
Sound	Ve, Vo, Kaa, Kee

Marriage on this Nakshatra. Intial problems in marriage and problems.

Chandra & Tara (Wife of Bhrihaspathi)

Tara was the wife of Brihaspati, the guru of Devas.  It required extra-ordinarily long hours of working (sometimes even 25 hours a day), chanting of mantras and developing strategies to combat the Asuras. Chandra eloped with tara. He is not going to return the wife of brithaspathi. She willing to came here. There was a battle between the gods.

Bhrama was settled the war to send back the tara. Finally Chandra agreed to go back. But at that time tara was already pregnant. The illegitimate child born bhudha. After that bhrihaspathi accepted and adopted as own child.

 

Bhramma & Sarawasthi

Bhraman created sarawsti. She take form of dear. He taken form of stag and searching for her. Rudra came to play and chopped the 5th head of bhrama.

CHARACTERISTICS

Mrigashirsha Nakshatra : Mrigashirsha 23-20 Taurus to 6-40′ Gemini

General Characteristics : Spiritual intelligence and research abilities, primary motivation – spiritual liberation, lack firm or fixed views, capable yet timid, good speech and active habits.

Translation : Having the face of a deer.

Symbol : Antelope’s head

Animal symbol: Female serpent – sensual seductive nature.

Ruling planet: Mars

Nature: Deva (god-like)

Mrigashira Nakshatra 1st Pada: It makes the native educated, intelligent, passionate, creative and good explorer. Problems related to male progeny are indicated but he will have daughters. Marital life will be average. We can recognize the native by his/her piercing eyes and sharp nails.

With the Sun as the lord, they have the will, capacity and enthusiasm to finish any work undertaken.Their nature is that of a king. Undertaking tasks beyond their capacity they will struggle to complete it, and with effort too.

Mrigashira Nakshatra 2nd Pada: People born in this combination are practical, social, calculative and humorous. But if the Nakshatra is afflicted, it may make the native adulterous. Native may have keen interests in singing and music. They become good soldiers With Mercury as lord, the Mrigashira Nakshshatra native shines as an illustrious scholar with literary and cultural tastes. He will be interested in performing religious ceremonies, but may be a little short-tempered.

Mrigashira Nakshatra 3rd Pada: People born in this pada will be good explorer but have hyper suspicious, impulsive and freaky nature. They always need some good counselor or advisor in taking important decisions of life.This pada relates to the intellectual, argumentative, suspicious, flaky and whimsical part of this nakshatra.

Mrigashira Nakshatra 4th Pada: People born in this pada will be good explorer but have hyper suspicious, impulsive and freaky nature. They always need some good counsellor or advisor in taking important decisions of life. Just as every star has some good features, the names too assigned to the natives born under each pada give some good characteristics.

Enable GingerCannot connect to Ginger Check your internet connection
or reload the browserDisable in this text fieldEditEdit in GingerEdit in Ginger×Enable GingerCannot connect to Ginger Check your internet connection
or reload the browserDisable in this text fieldEditEdit in GingerEdit in Ginger×Enable GingerCannot connect to Ginger Check your internet connection
or reload the browserDisable in this text fieldEditEdit in GingerEdit in Ginger×Enable GingerCannot connect to Ginger Check your internet connection
or reload the browserDisable in this text fieldEditEdit in GingerEdit in Ginger×

Read More

Expensive Joke that Companies have to take in ....

Recalling yester years....

I was working for a Chemical Company, we had a boss, He was nothing less than a joke.

This is what happened once...

The storage system needed additional Storage, the Boss came and said we need to buy additional HDD, as usual, we found the best fit and said: "buy the same HDD model and specification for compatibility with the existing set of disks ie 350MB *6, 5500RPM ".

The Boss came back, after a few days and said, I have got a good deal on the HDD purchase and said, "I have purchased 1TB, 7500RPM *6, see how smart and intelligent I am, you guys should learn from me".

We said : "WoW, Boss you are smart, you got a super deal", but we were laughing our butts off.

Later the IBM storage engineer came to fix the new HDD and said: I can't use this HDD, it won't work here, Boss was perplexed and was very angry with the IBM engineer.. and started shouting at him, The engineer left the place and said please talk to my office on how to fix this HDD.

The technical issues is when you have a RAID storage system in place all the HDD has to be with the same specification for the system to function and this was not known to our him. The HDD was an expensive one and could not be used anywhere else, Now he had to give an explanation to the management why he spent Rs.3.5L for the HDD when it was not needed.

This is what happens when you think you are smart but in reality, you are dumb

This is a real story from my career

Enable GingerCannot connect to Ginger Check your internet connection
or reload the browserDisable in this text fieldEditEdit in GingerEdit in Ginger×

Read More

Digital Transformation Strategy

Digital Transformation Strategy

Read More

Monkey

Read More

Information Technology Free online Consulting

I saw many times MD's and directors of companies not able to understand what the IT guys are saying and most have a complete inefficient IT system in place, there are some companies with claim to have fugal IT department and were proud of it. A closer look at the organisation setup relieved that it was far too expensive to operate and the term fugal was wrongly presented. By trusting their gut feeling ie. that nothing has happened for last couple of years, no redundancy was built all were single point failure. yes if you do this the investment will be deferential 50% of the full redundant  system.

If you have IT problem and not clear how to plan for the future with clear understand and a timed plan of action, I would suggest you to reach me for free IT consulting , I would like to introduce my self as Arvind GK and I'm an independent IT Consultant with 20+ years of experience, I have worked in the capacity of General Manager IT in big group and worked with organizations to name a few like TVS Fastener, TVS Tyres, Natesan syncrocone, Mittal Steels, Sunup Designs in USA, Astro Televison in Malaysis, Indovision in Indonesia, AskyB in US, Orchid Pharma, Danish Industrial confederation etc. I have handled IT with various improvement with results including Process improvement, ERP implementation, Operation etc, . I would be a ideal candidate if you are interested in saving money, increase in revenue and profit, you will sure get a great insight when you look at my profile at View Arvind GK's profile on LinkedIn https://in.linkedin.com/in/gkarvindr

“Putting IT together” – IT help for businesses – Concentrating on consultancy, Helping to make business goals and dreams more achievable with modern IT technology – Facilitating better use of existing staff and technology – Advising, developing, installing and training on Dynamic CRM websites, Android Apps, PC business software, SQL databases, Microsoft Office and improved documentation / procedures.

Here are some of the zones I can work with you:

Business advancement
IT Audits
IT Security
ERP like SAP
Android Apps or other application
Electronic Business (E-business, online business, etc)
Email solutions
Ticketing System - to keep track of your customer sales and service request
Process improvement (ITIL)
DR Plan and Implementation
Make a Road Map for IT in connection to you Business
System Design and Planning, Security Solutions
Fittings and Software Solutions
HW/SW Procurement
Datacenter Build (Consulting and Execution )
Remote Access Solution
Information Backup and Recovery Solutions
Report Imaging (Electronic Records/Go Paperless) Solutions
Preparing Service Contracts
Training


We could talk about this or you direct me to the right person to discuss about this, so we can explore something valuable to incorporate into your organisation.

you can also see my professional profile at https://in.linkedin.com/in/gkarvindr

Read More