Rsync: commands of Linux on local & remote host.

rsync is a free software computer program for Unix and Linux like systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate.

Installation :

yum install rsync

rpm
–ivh
rsync.*

rsync command common options:

  • delete : delete files that don’t exist on sender (system)
  • v : Verbose (try -vv for more detailed information)
  • e “ssh options” : specify the ssh as remote shell
  • a : archive mode
  • r : recurse into directories
  • z : compress file data

Same local computer:

rsync -av /source/folder /destination/folder

From local to remote computer:

rsync –progress –partial –avz /folder/to/copy/ user@remote.server:/remote/folder

 

 

From remote to local computer:

 

rsync --progress --partial -avz user@remote.server:/folder/to/copy/ /local/folder

Synchronize two folders with rsync:

From one folder to another on the same server:

 

rsync -av --delete /source/folder /destination/folder

From local to remote computer:

rsync -avz –delete /source/folder user@remote.server:/destination/folder

From remote to local computer:

rsync -avz --delete user@remote.server:/source/folder /destination/folder

backup your files to an external drive, this what I usually do:

rsync -av --delete /home/guillermo/ /media/disk/backup-guillermo/

 


 

Setup of Rsync for Windows Using DeltaCopy

As we know how powerful rsync utility is on the Linux platform.

If we use the standard copy feature of Windows, we won’t be able to incrementally copy only the changed blocks.
So ,
Deltacopy is a Windows software that is a wrapper for the rsync program. We don’t need to install cygwin to get Deltacopy working. Deltacopy is an open source software.

1. Download and Install DeltaCopy

First, http://www.aboutmyx.com/files/DeltaCopy.zip and install it.

2. Register DeltaCopy a Windows Service

Once it is installed, launch the DeltaCopy Server, which will display the DeltaCopy Server Console window. Click on the ‘Register Windows Service’ to register this as a windows service. At this stage, the “Start Server”, and the “Stop Server” button will be disabled.


This will prompt you to enter your windows username and password to register the service as shown below.


Once you enter the windows username and password, it will display the message “Service created successfully”.

3. Start DeltaCopy Server

Now the “Start Server” button will be enabled. Click on this to start the DeltaCopy Server.

You might get “Could not start the service. The service could not be logged on. Check user id and password specified for login” message, if you’ve specified invalid credentials. You also might want to run this service as “Local System” to fix this issue.

Go to “Control Panel” -> Administrative Tools -> Services -> Select “DeltaCopy Server” -> Right mouse click -> Select “Properties” -> Go to “Log On” tab -> Select “Log on as: Local system account”, as shown below.


Now, click on “Start Server” from the DeltaCopy Server Console, which should start the server. Once the server is started the “Stop Server” button will be enabled as shown below.


4. Create Virtual Directory

Now, let us set up the virtual directory on the server side, which will be used by the client, when they connect to it to perform rsync.

From the DetalCopy Server Console, click on “Virtual Directories” tab -> Double-click on the “”, which will display the following window, specify an alias name for this virtual directory. In this example, I gave “MyPictures” as the alias name.


Now, select the “MyPictures” alias name, and specify a corresponding directory for this alias, by clicking on the ‘Browse’ button in the path. I selected the “C:\Users\vaibhav\Pictures”.

If you want clients to specify a username/password to access this particular virtual directory, click on “Use Authentication” check-box, and specify a username password. Whatever username and password combination you specify here should be used from the DeltaCopy client.

5. Setup DeltaCopy Client

Typically you’ll install DeltaCopy server on one Windows system, and the DeltaCopy client on another windows system, and perform rsync from Windows client with Windows server.

However, for testing purpose, let us launch the DeltaCopy client from the same machine where the server is installed and see how it works. When you installed the DeltaCopy as explained above, it installs both server and client.

Launch the DeltaCopy client, and click on “”, which will display the following window.


  • Enter “myclient” in the profile name.
  • Enter “localhost” for the server name.
  • Click on the “…” button that is located next to the “Virtual Directory Name” drop-down list, this will connect to the server (in this case localhost), and get all the available virtual directories and populate the drop-down list. You’ll see the “MyPictures” virtual directory that we created from the previous step, select that.
  • Click on “Test Connection” button to make sure everything is working properly.
  • Finally, click on “Add Profile.

For testing purpose, create a folder “C:\temp” and add a test file in this folder. We are going to sync this folder (assuming that this is from client), to the “MyPictures” profile on the server (which is “C:\Users\vaibhav\Pictures” on the same machine in this example).

Click on “Add Folder” and select “C:\temp”. If you want to add individual files to this “myclient” profile, click on “Add Files”.

If you want to execute this “myclient” profile (i.e to perform this rsync) at certain interval, click on “Modify Schedule” and set your schedules accordingly. This is the standard Windows scheduling functionality.

If you want to execute this immediately, select the “myclient” profile -> Right mouse click -> and select “Run Now” as shown below.


This will pop-up the status window as shown below, which indicates that this called the rsync.exe to execute the rsync and the files for successfully synced. i.e The “C:\temp” folder is now copied over (synced over) to “C:\Users\vaibhav\Pictures”.


In this article we showed how to perform rsync on the same machine. However, as explained earlier, you can install DeltaCopy Server on one system, and DeltaCopy Client on another system, and perform rsync between Windows server using the same procedure as explained above. You can also sync from Windows to Linux by connecting from the DeltaCopy client that is running on Windows to a remote Linux server.

VIMP(Video Library) installation & configuration on Ubuntu 12.4 – i386 server :

Requirements

To install ViMP, you need a dedicated server running any Linux operating system (Debian Squeeze recommended), a web server (Apache2), PHP and a MySQL database.

We recommend the following minimum hardware configuration:

  • Dual-Core processor or better with at least 2.5 GHz
  • At least 4 GB RAM
  • At least 250 GB Disk space (SATA-II or SAS). For additional data security, we recommend using a RAID controller.

 

The following software is required to run ViMP:

 

  • Apache2: Version 2.2 recommended & mod_rewrite must be enabled
  • PHP: At least version 5.2.7 & GD, mysql, XSL and cURL extensions must be installed &

    PHP CLI (Command Line Interface) must be installed

  • MySQL: At least version 5.0 required & InnoDB storage engine is required
  • Transcoding: MEncoder / MPlayer 1.0RC2 or newer ,FFmpeg ,Additional codec packages & Ruby and FLVTool2

     

Preparing the server:

Update server (not used on production servers)

apt-get update

apt-get upgrade

apt-get dist-upgrade

To install Apache2, simply run:

apt-get install apache2

Now, enable mod_rewrite:

a2enmod rewrite

/etc/init.d/apache2 restart

To install PHP5, run:

apt-get install libapache2-mod-php5

Then, install the additional packages:

apt-get install php5-cli php5-mysql php5-gd php5-xsl php5-curl

 

To install the MySQL database server, run the following commands:

apt-get install mysql-server

When prompted for a root password, Make sure to remember the password or write it down in a safe place, you’ll need this password later.

Transcoding: First, add the additional multimedia packages to the package source list of the operating system. Therefore create a configuration file in /etc/apt/sources.list.d:

 

touch /etc/apt/sources.list.d/debian-multimedia.org.list

Once the file is created, open it with an editor and add the following lines:

# Debian Multimedia Repository

deb http://www.debian-multimedia.org stable main non-free

deb-src http://www.debian-multimedia.org stable main non-free

deb ftp://ftp.debian-multimedia.org stable main non-free

deb-src ftp://ftp.debian-multimedia.org stable main non-free

Next, the keyring of debian-multimedia.org has to be installed:

apt-get update

apt-get install debian-multimedia-keyring

Now install the packets of the transcoding software:

apt-get install mplayer mencoder ffmpeg flvtool2

Next, let’s install the essential codec package for MEncoder. This package provides basic binary codecs for video decoding and encoding. Please check what kind of architecture (32 Bit or 64 Bit) your system has and only install the package for your architecture.

Installation on 32-Bit systems:

apt-get install w32codecs

Configuration:

Once all packages are installed, some configuration settings need to be adjusted.

Apache2

If you are planning to install your video website in a sub-directory (e.g. http://http://www.myportal.com/community) skip this section and follow the instructions in the appendix section Installing ViMP into a sub-directory.

We recommend creating a new virtual host in a separate directory for this installation. Therefore we will first create the directories:

mkdir -p /var/www/myportal/data

mkdir -p /var/www/myportal/logs

chown -R www-data:www-data /var/www/myportal

Now create the configuration file for this virtual host:

cd /etc/apache2/sites-available

touch
myportal

Once you have created the file, open it in your favorite editor and create the VirtualHost entry:

<VirtualHost *:80>

DocumentRoot “/var/www/myportal/data/web”

ServerName
myportal.com

ServerAlias http://www.myportal.com

<Directory “/var/www/myportal/data/web”>

AllowOverride All

RewriteEngine On

Options -Indexes FollowSymLinks

</Directory>

ErrorLog /var/www/myportal/logs/error.log

CustomLog /var/www/myportal/logs/access.log combined

LogLevel error

</VirtualHost>

Save the file and close the editor. We will now enable the virtual host:

a2ensite myportal

/etc/init.d/apache2 reload

PHP

First, adjust the values for the PHP Apache2 module. Open /etc/php5/apache2/php.ini in your favorite editor. Then locate the following entries and change their values to the indicated values:

  • upload_max_filesize: 1024M
  • post_max_size: 1024M
  • memory_limit: 512M

     

Once you are done, save and close the file. Then change the settings for PHP CLI by opening /etc/php5/cli/php.ini and change the values as indicated above.

When you are done, restart Apache2 to load the new configuration:

/etc/init.d/apache2 restart

MySQL

First, let’s create a database for the community:

mysqladmin -p create myportal

When prompted for a password, enter the root password you have chosen while installing MySQL.

Next we’ll create a designated user for this database. Therefore enter the MySQL console:

mysql -u root –p

Next, enter the following SQL queries. Please note that all queries must be terminated with a semicolon:

mysql > CREATE USER myportal@localhost IDENTIFIED BY ‘<password>’;

GRANT ALL PRIVILEGES ON myportal.* TO myportal;

FLUSH PRIVILEGES;

mysql >exit

Please replace <password> by a strong password and remember it. It will be needed later during the installation.

Installation

Now that all programs are configured, we can install ViMP. Therefore the ViMP download package needs to be on the server. Copy the archive containing ViMP into the directory we have created before (/var/www/myportal/data). Then, extract the archive:

cd /var/www/myportal/data

tar xvzf vimp.framework-<version>.tar.gz

This will extract all files into the current directory. Once you are done, delete the archive:

rm vimp.framework-<version>.tar.gz

If you have modified ViMP, copy all files to the /var/www/myportal/data directory.

Then make the symfony file executable:

chmod +x symphony

Now
Installation of the SourceGuardian PHP extension:

We assume that a virtual host has been set up and points to the directory /var/www/myportal/data.

Extract the PHP extension to a temporary directory:

mkdir /tmp/loader

cd /tmp/loader

tar xfz /var/www/myportal/data/data/loader.tar.gz

Now the PHP extension is located in this directory, including different files for different operating systems. Select the operating system and PHP version that fits for your server and copy the file into the lib directory of PHP:

cp Linux_x86-32/ixed.5.2.lin /usr/lib/php5/20060613+lfs/

Next you need to tell PHP, that it must load the extension. Therefore create a configuration file in /etc/php5/conf.d:

cd /etc/php5/conf.d/

touch sourceguardian.ini

Once the file is created, open it with an editor and add the following lines:

[sourceguardian]

zend_extension=/usr/lib/php5/20060613+lfs/ixed.5.2.lin

Save the file and close the editor. Then restart Apache2:

/etc/init.d/apache2 restart

Next, run the framework:init
task by adding the above DSN:

./symfony framework:init mysql://myportal:<password>@localhost/myportal

After that you need to set up the cronjob which converts uploaded videos into the needed formats. Edit the crontab:

crontab –e

And add the following line at the bottom:

*/1 * * * * /var/www/myportal/data/scripts/sync_new_uploads.sh > /dev/null 2>&1

This will execute the sync script every minute. If no new media files are found, the script simply terminates without any further actions.

Basic usage

The installation is complete now. Point your browser to http://http://www.myportal.com to see your video website in action.

The in it task automatically creates three users representing the three access roles:

  • admin (password: admin) as administrative user
  • moderator (password: moderator) as moderator user
  • user
    (password: user) as basic user

Ms-deploy (web-deploy) to Sync Content folders & Drive with any port:

As the task assign to me to search and find out the Sync the E drive of Solar toward the solar DR over the internet . I have found and test the so many tools like (FreeFileSync,Allway Sync,SyncToy,BestSync,dropbox , Dirsyncpro & DSynchronize)

But after discussing with Prasahnt sir and reading blog of Msdelopy ,I have been playing around with it. With some problems like we have to use other port than 80 and other user as they have used for IIS sync but we have to use for drive ( folder ) sync.

So below is the commands wich I have used to install by silent mode to install Msdeploye

msiexec /I <path of webdeploye.msi> /passive ADDLOCAL=ALL LISTENURL=http://+:8888/MsdeployeAgentservice2

After open the port on both firewalls I have test the folder sync by using this command this is modified command to be used by using different port the 80

msdeploy -verb:sync – source:contentPath=E:\softwares,computername=http://180.179.68.145:8888/MsDeployAgentService2,username=<myusername>,password=<mypassword> -dest:contentPath=E:\softwares

And for general (port 80) we can use as like below.

msdeploy -verb:sync -source:contentPath=E:\softwares,computername=180.179.68.145,username=<myusername>,password=<mypassword> -dest:contentPath=E:\softwares

So created bat file & schedule the job as per given snapshot for IIS folder sync,All glassfish domain sync and E Drive sync.


 

 

 

 

 

 

 

 

 

 

And at finally we can find the all E drive folders are modified date wise from the solarex1 E drive.


 

 

 

 

 

 

 

 

 

 

 

Thanks J !