Moving Email from CS Mail Server – dragon.rutgers.edu

by Hanz Makmur, Mar 2016.

Note: This is an old document describing how to move email from ‘dragon.rutgers.edu”.  This same instruction can also be used to move email  out from your current email system when you graduated of Rutgers assuming you have IMAP enabled on both systems.  

IMPORTANT: As of Jan 2020,  Google started to enforce oAuth authentication for IMAP. If your account is already forced to use XOAuth2, this instruction is not applicable.  You can follow Gmail and XOAUTH2 specific instruction
Otherwise, please follow instruction on  Migrating Email Out of Rutgers Connect.

As part of the President’s Strategic Plan for Rutgers University, Computer Science is moving to Rutgers Connect, a Microsoft 365 based Cloud Mail server. Details Information about Rutgers Connect has been posted and Official Communications on email migration was sent in Feb 8, 2016.

OIT manages the email migration for current users and have sent email to those affected asking them to respond by Mar 1, 2016. Unfortunately many CS Mail users who do not have active status must move their email out themselves by end of 2016.

Please see CS Email Transition To Rutgers Connect on how you are affected and what steps you must take to avoid loosing email.

This document is written to help users those who prefer to move their email themselves or those who do not qualify for OIT managed email migration.

In this document you will learn how to move your existing emal on CS mail server to other IMAP based email server. If you don’t have a lot of email, a manual process may be the easiest way to follow. Otherwise, you may need to consider the automatic process.

1. Manual process

Manual process can be done on your modern email client (like AppleMail, ThunderBird). It is an easy process to do but it is the most time consuming. If you have a lot of email and folders of email you may want to consider the Automatic process.

Follow steps below to move your email manually:

1. Setup your email client to read email from both dragon.rutgers.edu and ScarletMail or Rutgers Connect. Please follow instructions provided by the respective email system on how to setup your modern mail client with that system.
2. If you have email folders, you will need to create individual folders on the ScarletMail or Rutgers Connect end.
3. Drag email from each dragon.rutger.edu folder to the newly created folder on the ScarletmMail or Rutgers Connect end.
4. Repeat until finish.

2. Automatic Processs Using IMAPSync

Migrating all your email can be done in 2 ways using IMAPSync program. If you still have access to CS Linux account, you can use CS resources or you can install the software yourself on your own computer.

A. Using CS iLab/Grad/Faculty Linux Computer

If you need temporary CS Linux account – only until Dec 2016,  please let us know and we will try to setup a guest account in late August or early September 2016 for you to help migrate your email.

B. Using Your Own Computer

On LinuxOS, simply installed imapsync via yum or apt or whatever package manager you run on that LinuxOS.

On a MacOSX, you can easily install imapsync via Homebrew command: brew install imapsync. Or you can also install it viaFinkProject

On a WindowsOS, you need to download the source, Strawberry Perl and follow Part B of the instruction

If you dont want to deal with putting it together, you can also purchase ready made one at http://imapsync.lamiral.info/

 

Migrating Email Using IMAPSync

Pre-requisites:

1. Enable IMAP on your ScarletMail and create a separate password for it. Please follow instruction provided by ScarletMail on how to enable IMAP protocol.
    
2. Set a separate password to access ScarletMail/ScarletApps via mobile/IMAP/POP. Please visit the Manage Your ScarletApps Mobile/IMAP/POP Password tool at https://netid.rutgers.edu.
    
3. Optional: IMAPSync runs on a terminal and takes a long time. You may want to consider using Linux program knows asscreen. See a quick how to use screen so you can keep your procces running despite dropped connection.

Syncing your Email from CS Mailserver (dragon.rutgers.edu) to Other System.

The imapsync command may need to be repeatedly run until the transfer is completed. It basically does one-way sync from CS mailserver a destination server. It can be used on any IMAP based server of your choice.

You can start migrating your email now if you already have a new email account somewhere else and you can repeatedly run this command until all your email are moved.

Two examples below show you what you need to do to sync to ScarletMail and Rutgers Connect mail servers. Please note that it will take a long time to complete. Due to limitation placed by the destination server, it may terminate after a certain hours and you will need to re-run to complete your sync. It can take 2-3 or more syncs to transfer 5 GB worth of email. Please look at yourmigration.log to see the status of your email migration.

Syncing To ScarletMail

Below is the command for synching email to Scarletmail from CS Mailserver (dragon.rutgers.edu). Please pay attention on the redcolor line below. You will need to substitute it with you own information. For more info seehttps://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt for more options.

/usr/bin/imapsync \
--pidfilelocking \
--noreleasecheck \
--maxsize 45000000 \
--maxlinelength 10500 \
--maxbytespersecond 10000 \
--noexpunge \
--disarmreadreceipts \
--addheader \
--regextrans2 "s/[ ]+/_/g"  \
--regextrans2 "s/['\^\"\\\\]/_/g" \
--regexflag 's/\\Flagged//g' \
--logfile migration.log \
--host1 dragon.rutgers.edu \
--timeout1 180 --ssl1 \
--user1  your_netid \
--passfile1  Name_of_file_containing_your_CS_password \
--host2  imap.gmail.com \
--timeout2 180 --ssl2 \
--user2  your_netid@scarletmail.rutgers.edu \
--passfile2  Name_of_file_containing_your_scarletmail_imap_password \
--subfolder2 Name_of_sub_folder_to_store_your_dragon_email_to

Note: If you are using Google 2 steps verification, use your app_specific password for passfile2

Syncing To Rutgers Connect

Below is an example command for synching email to Rutgers Connect from CS Mailserver (dragon.rutgers.edu). Since Rutgers Connect essentially your new @cs mailserver, the destination username will your_netid@cs.rutgers.edu . Please pay attention on the red color line below. You will need to substitute it with you own information.

/usr/bin/imapsync \
--pidfilelocking \
--noreleasecheck \
--maxsize 45000000 \
--maxlinelength 10500 \
--maxbytespersecond 10000 \
--maxmessagespersecond 4 \
--noexpunge \
--disarmreadreceipts \
--addheader \
--regexflag 's/\\Flagged//g' \
--regextrans2 's/[ ]+/_/g' \
--logfile migration.log \
--host1 dragon.rutgers.edu \
--timeout1 180 --tls1 \
--user1 your_netid \
--passfile1 Name_of_file_containing_your_CS_password \
--host2 outlook.office365.com \
--timeout2 180 --tls2 \
--user2 your_netid@cs.rutgers.edu \
--passfile2 Name_of_file_containing_your_rutgers_netid_password

/usr/bin/imapsync --help

For example if you like to sync only certain folders, you can add these option to the end of the command above: 

    --folder       str  : Sync this folder. Example:  --folder "Summer2014"
    --folder       str  : and this one, etc to sync more folders

    --folderrec    str  : Sync this folder recursively.
    --folderrec    str  : and this one, etc to sync more folders

    --folderfirst  str  : Sync this folder first. Example:  --folderfirst "Work"
    --folderfirst  str  : then this one, etc to sync more folders

    --folderlast   str  : Sync this folder last. Example: --folderlast  "[Gmail]/All Mail"
    --folderlast   str  : then this one, etc to sync more folders