Linux, Python
Using procmail with custom python script
If you want to execute some special commands, log or make an API call when you’re receiving an email onto your server, you can easily set up this by using procmail. Procmail is designed to filter and sort emails but can make any call you want.
Here, as an example, we will set up a simple call to a python script that will read the content of the mail (headers and body) and put the information into a log file.
1. Install procmail
Depending on the OS you’re using, you should find a package pre-compiled in the common repositories.
For example, on a Debian-based:
|
1 |
apt-get install procmail |
or on a CentOS-based:
|
1 |
yum install procmail |
2. Build the python script you want to call to analyze the message
You now have to prepare your script you will execute when receiving an email, that will read and parse the content to log interesting information in a file.
Let’s create a script called procmail_script.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
import os import time import email import sys import logging.handlers import base64 # Constants LOG_FILE_PATH = os.path.expanduser('/opt/mailAnalysis.log') # Set up a logger my_logger = logging.getLogger('MyLogger') my_logger.setLevel(logging.INFO) handler = logging.handlers.RotatingFileHandler(LOG_FILE_PATH, maxBytes=500000, backupCount=4,) formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s", "%Y-%m-%d %H:%M:%S") handler.setFormatter(formatter) my_logger.addHandler(handler) # Main function def main(): try: # Get message full_msg = sys.stdin.read() msg = email.message_from_string(full_msg) # Prepare dict containing data data = {} # Fill dict data['From'],e = email.Header.decode_header(msg['From'])[0] if msg.has_key('From') else '' data['To'],e = email.Header.decode_header(msg['To'])[0] if msg.has_key('To') else '' data['Subject'],e = email.Header.decode_header(msg['Subject'])[0] if msg.has_key('Subject') else '' data['Body'] = msg.get_payload() # Add information to log my_logger.info('From: ' + data['From']) my_logger.info('To: ' + data['To']) my_logger.info('Subject: ' + data['Subject']) my_logger.info('Body: ' + data['Body']) except Exception,e: my_logger.error('----- ERROR ENCOUNTERED') my_logger.error(str(e)) my_logger.error('----- END OF ERROR') # Main program if __name__ == "__main__": start_time = time.time() my_logger.info('----- START: ' + str(time.time())) result = main() my_logger.info('----- END: ' + str(time.time())) my_logger.info('Elapsed Seconds: ' + str(time.time() - start_time)) handler.close() sys.exit(result) |
3. Configure your user to pass the mail to your script
So that you can pass rules you want to execute when receiving an email, you need to create a file (hidden) called .procmailrc that will take place in the home directory of the user you want to use.
For example, for executing rules when receiving email to [email protected], you will have to put that file into the home dir like /home/mailuser/.procmailrc
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
LOGFILE=/var/log/procmail.log VERBOSE=YES :0: * ^Subject:.*procmail.* * ^TO_mailuser@mycompany.(com|net) { :0c procmail-dir/ :0 fw | /usr/bin/python /home/mailuser/procmail_script.py :0 e procmail-failed-dir/ } |
This will perform multiple steps:
- Check that the mail recipient is [email protected] or [email protected]
- Put a copy of the email into the procmail-dir directory
- Pass the message to our python script procmail_script.py
- Discard the message if the script succeeds (remove from queue) or copy it to procmail-failed-dir if failed (so you can process it later)
4. Prepare an email and perform a testing locally
First, create a sample mail that you will use for testing in a file called procmail_test.txt:
|
1 2 3 4 5 6 7 |
From: me@mycompany.com To: mailuser@mycompany.com Subject: This is a procmail testing Hey there, I hope this message will be parsed and logged properly as expected. This is my first test for procmail deployment! |
Then, you can test it by executing procmail manually:
|
1 |
procmail VERBOSE=on /home/mailuser/.procmailrc < /home/mailuser/procmail_test.txt |
Now, open the file /opt/mailAnalysis.log and you should have something like:
|
1 2 3 4 5 6 7 8 9 |
2018-03-31 08:08:45 - INFO - ----- START: 1522570125.06 2018-03-31 08:08:45 - INFO - From: me@mycompany.com 2018-03-31 08:08:45 - INFO - To: mailuser@mycompany.com 2018-03-31 08:08:45 - INFO - Subject: This is a procmail testing 2018-03-31 08:08:45 - INFO - Body: Hey there, I hope this message will be parsed and logged properly as expected. This is my first test for procmail deployment! 2018-03-31 08:08:45 - INFO - ----- END: 1522570125.12 2018-03-31 08:08:45 - INFO - Elapsed Seconds: 0.0624470710754 |
Linux
Execute cron on a specific day of the month (e.g. second monday)
How to execute a cron on a specific day of the week once in the month?
This could look simple as we could think that this line in cron would do the trick:
|
1 2 |
# Run on every second Tuesday of the month 15 3 8-14 * 2 /usr/bin/bash /opt/myscriptfortuesday.sh |
But this would not work as the ‘2’ for checking the Tuesday will come as a OR condition, and the command would be executed from day 8 to day 14 and on every Tuesday of the month.
As a workaround for that, you can use that command:
|
1 2 |
# Run on every second Tuesday of the month 15 3 8-14 * * test $(date +\%u) -eq 2 && /usr/bin/bash /opt/myscriptfortuesday.sh |
Here is the explanation of this cron line:
|
1 2 3 4 5 6 |
15 = 15th minute 3 = 3am 8-14 = between day 8 and day 14 (second week) * = every month * = every day of the week test $(date +\%u) -eq 2 && /usr/bin/bash /opt/myscriptfortuesday.sh = the command to execute with a check on the date |
Doing this check will allow to verify first that we are on the second tuesday before to execute the command. Don’t forget to add a backslash before the ‘%’ character to escape it.
Linux
Control services on Linux (systemV, systemd, initctl…)
On Linux, there’s many ways to control services that can run on your system. Here are the main and most known init systems that you can find on the common distros, depending on their version.
SystemV
That system is one of the oldest and one of the most common. Its init scripts are stored in /etc/init.d/
To list all the available services, you have to use that command:
|
1 |
service --status-all |
To perform an action on one of those services, you will be using:
|
1 |
service PROCESS_NAME (start|stop|restart|status) |
Upstart (initctl)
Upstart is a successor of SystemV scripts. It works asynchronously, and its scripts are stored in /etc/init/
To list all the available scripts, you have to use that command:
|
1 |
initctl list |
To perform an action on one of those services, you will be using:
|
1 |
initctl (start|stop|restart|status) PROCESS_NAME |
SystemD
Its name SystemD means System Daemon. It manages daemons that can be running on a system. It’s a successor of upstart and allows a more flexible management for the services. Init scripts are stored in /etc/systemd/system/
To list all the available scripts, you have to use that command:
|
1 2 |
systemctl --list-units systemctl --list-unit-files |
To perform an action on one of those services, you will be using:
|
1 |
systemctl PROCESS_NAME (start|stop|restart|status) |
SupervisorD
Finally, SupervisorD is a supervisor focusing mainly on the applications more than the system. It allows management of applications execution and control their life like you could do with system services.
To display all the applications managed by supervisord:
|
1 |
supervisorctl status all |
To control those processes, you will be using:
|
1 |
supervisorctl (start|stop|restart|status) PROCESS_NAME |
Linux
Generate a CSR with openssl
Generate a CSR (Certificate Signing Request) on your server when you want to get a certificate from a certified provider is often a mandatory step, very easy to execute.
Here are the different steps to execute:
- Create a specific directory where you will put all your files
1mkdir sub.domain.com && cd sub.domain.com - Generate a private key of 2048 bits
1openssl genrsa -out sub.domain.com.key 2048 - Now generate a CSR with openssl and with the private key you just generated
1openssl req -new -sha256 -key sub.domain.com.key -out sub.domain.com.csr
Many information will be asked during the creation:
123456789Country Name (2 letter code) []:State or Province Name (full name) []:Locality Name (eg, city) []:Organization Name (eg, company) []:Organizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:Email Address []:A challenge password []:An optional company name []: - You now have your CSR and your private key
It’s up to you to get your signed certificate from an official provider using those files.
Linux
Change or remove password expiration for linux user
It can happen that you’re getting that message when trying to connect to your linux server:
|
1 2 3 4 5 |
You are required to change your password immediately (password aged) WARNING: Your password has expired. You must change your password now and login again! Changing password for user mylinuxuser. (current) UNIX password: |
The message is quite explicit and you need to update your password right now.
If you don’t want to update your password too many times, you can update the frequency of the expiration to 90 days for example:
|
1 |
chage -M 90 mylinuxuser |
Or you can completely disable the expiration by pushing the max value for expiration date to 99999 days:
|
1 |
chage -m 0 -M 99999 -I -1 -E -1 mylinuxuser |