Solaris – Sendmail Troubleshooting Reference

Know Everything about Solaris Sendmail Troubleshooting

The Difference Between the Sendmail Daemon in Solaris 9/10 and Prior versions

With the release of Sendmail 8.12.1, significant changes were made to how Sendmail is installed and used. Historically Sendmail was installed as a set-uid binary that was owned by user root. Due to the security risks of such a configuration, sendmail was modified so that it could run as a set-gid binary and thereby reduce the security risks. At the same time the roles of sendmail were more clearly defined and separated into two seperate modes, depending on the options passed to the sendmail binary.

In Solaris 8 and prior versions has one mail queue:

– /var/spool/mqueue

In Solaris 9 and above, sendmail uses two  queue  structure, as below :

– /var/spool/mqueue  managed by “/usr/lib/sendmail -bd -q15m” daemon owned by root.
– /var/spool/clientmqueue managed by “/usr/lib/sendmail -Ac -q15m” which is owned by smmsp
This is to make sendmail more secure by not giving out setuid for root in the binary.

There two roles for sendmail:

  • Mail submission program ( MSP )
  • Mail transport agent ( MTA ).
An email message is normally written in a mail program, or mail user agent ( MUA ), such as pine or mailx. When the email is sent, it is passed to a MSP. The MSP processes the email message and then passes it on to a MTA for further delivery. By default on Solaris 9 and 10, this MTA is the MTA running on the local host.
Each sendmail role now uses it own configuration file and spool directory. While this adds more complexity to the sendmail environment and configuration, it results in a more secure system and finer control of sendmail.

Mail Transport Agent

By default, sendmail runs on the local host in daemon mode and listens for incoming SMTP connections from other hosts and the mail submission program running locally. This daemon (/usr/lib/sendmail -bd -q15m) is run as root.
In daemon mode, sendmail is acting as a MTA  as per the configuration made in  `/etc/mail/sendmail.cf’ .
Whenever  the MTA is unable to forward the email to remote MTAs, it will spool email messages in `/var/spool/mqueue’ ( only accessible by root)
 

Mail Submission Program

Sendmail, as a MSP, will accept email and will try to pass it to the sendmail daemon running on the local host. If the MSP is unable to pass along the email it will be queued in in `/var/spool/clientmqueue.’ ( owned and writable to the group “smmsp”).
A seperate sendmail process (/usr/lib/sendmail -Ac -q15m) runs  to check that spool directory every fifteen mintues and attempt redelivery of any email messages queued.

Sendmail  acts as MSP as  per the configuration made into `/etc/mail/submit.cf’ .   Both the files  `/etc/mail/submit.cf’ and `/etc/mail/sendmail.cf’ share a common syntax and most sendmail configuration options can be used in either configuration file.

The group is smmsp and is the same group that owns the sendmail binary. Since sendmail is set-gid it will be able to write to this directory when it is called as a MSP. If sendmail becomes vulnerable to a local exploit, the scope will be restricted to whatever the group smmsp has access to, in this case the contents of the spool directory.
Sendmail in solaris 9/10 can be configured to run in one of the two modes mentioned below:
Daemon Mode :    By default, Sendmail on Solaris 9 and 10 runs the two processes mentioned above, this is called Daemon Mode.
Mail Submission Program Mode  –   In this mode only acts as mail submission program, but there are limitations to doing this.
NOTE: Please be advised that the direct editing of the submit.cf and the sendmail.cf files is not recommended nor supported.  Changes should be made to the respective M4 macro configuration (.mc) files located in /usr/lib/mail/cf/ and the respective configuration files regenerated as per the details in /usr/lib/mail/README.
Rest of this post focus on Troubleshooting Sendmail issues while Sending and Receiving mails

Sendmail Troubleshooting for Mail Sending Issues

 

 

Below are the Guidelines to Diagnosis and Troubleshoot main sending issues in sendmail.

 

Steps to ensure that your mail client setup is correctly configured

Step 1. In the configuration – make sure that your outgoing mail server is pointing to the right mail server that you intended to use.
Step 2. Verify that the computer you are trying to send email from is not running a firewall that may be interfering with SMTP traffic on port 25. Ensure that if you are behind a firewall, that it is not blocking the network traffic between your mail client and the mail server.
Step 3. Verify if other similar configured mail clients are able to send out mail. If so, try to find out the differences with yours and if needed correct and retry.
Step 4. If the underlying operating system has a telnet capable client, verify if a connection on port 25 (SMTP) of the mail server.
Check the Port with below command
# telnet mailserver.mydomain.com 25
 
Valid Sendmail server will answer with a following similar output:
 
220 mailserver.mydomain.com ESMTP Sendmail 8.13.8+Sun/8.13.8; Mon, 21 Apr 2008 11:36:17 +0200 (CEST)
 
When you type the ‘help’ command the available commands the mail server understands are listed:
 
214-2.0.0 This is sendmail version 8.13.8+Sun
214-2.0.0 Topics:
214-2.0.0       HELO    EHLO    MAIL    RCPT    DATA
214-2.0.0       RSET    NOOP    QUIT    HELP    VRFY
214-2.0.0       EXPN    VERB    ETRN    DSN     STARTTLS
214-2.0.0 For more info use “HELP <topic>”.
214-2.0.0 To report bugs in the implementation contact Sun Microsystems
214-2.0.0 Technical Support.
214-2.0.0 For local information send email to Postmaster at your site.
214 2.0.0 End of HELP info
 
Try the following to send an e-mail from the command line:
 
helo clienthostname.mydomain.com
250 clienthostname.mydomain.com Hello clienthostname.mydomain.com [xx.xx.xx.xx], pleased to meet you
mail from:myname@mydomain.com
250 2.1.0 myname@mydomain.com… Sender ok
rcpt to:recepientname@recepientdomainname.com
250 2.1.5 recepientname@recepientdomainname.com… Recipient ok
data
354 Enter mail, end with “.” on a line by itself
subject: This is a test
to:recepientname@recepientdomainname.com
This contains the BODY of a test mail
.
250 2.0.0 m3L9aH0W018247 Message accepted for delivery
quit
221 2.0.0 mailserver.mydomain.com closing connection
 
Note the dot (“.”) at the end of the mail body.
 
If this test succeeds (mail is delivered to it’s final destination), then most likely the issues is with the mail client configuration .
Step 5. If possible,  try to capture the network traffic (e.g. via “snoop”/”tcpdump”) on both ends while trying to send a mail form the mail client. Analyze if there is SMTP traffic between your client and the mail server,using any of the below available commands
– ethereal
– wireshark
– netcap
– tcpdump
– snoop

 

 

Steps to validate the network path between the mail client and mail server

Step 1. Verify correct client configured
If the user is trying to connect to a specific host then they must ensure that the ip address being used matches the system they are trying to connect to.
Users may use the generic system lookup tool, getent, to check for matches for their server.
# getent hosts myserver 
192.168.1.228 myserver
If the ip does not match what is expected then the user will need to change the entry. This will vary depending upon which name service is being used. LDAP, files, nis, nis+ or DNS.
Step 2. Verify network is up and configured properly 
To connect to remote host there needs to be a network interface plumbed and UP.
Step 3. Verify target server is up and reachable
To connect to remote host there needs to a network route from the client ip the remote host.
You can use commands like (ping, traceroute ) to establish if the remote host answer to any request
 #ping IPaddress or hostname
 #traceroute IPaddress or hostname
Note: The hostname response depends on what namingservices you uses (LDAP, DNS, NIS+ , NIS(YP) or local hostname /etc/hosts ). If the namingservice do not respond you can have issue with the hostname lockup ( see Step1)
Step 4. verify service is up and reachable 
The remote server needs to be up and having the correct services running to which the client can connect. This largely depends on the type of service.
The most common login is Secure Shell (SSH)
 
SSH Check
# svcs | grep ssh
online         Nov_15   svc:/network/ssh:default
Telnet Check
On most hosts the telnet service is disabled because of security matter. We only use the telnet while checking the port 25 on the server.
Step 5. Verify the packets are getting through on the network. 
If the the service is running, the network configured and a network route exists we need to ensure that the actual application packets are being delivered to the server.
To determine this we will need to capture network traffic from both the client and server.

 

 

Step 1:  Invoke /usr/lib/sendmail with the -d0.1 debug flags.
Example:
$ /usr/lib/sendmail -v -d0.1 < /dev/null
Version 8.13.8+Sun
Compiled with: DNSMAP LDAPMAP LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8
MIME8TO7 NAMED_BIND NDBM NETINET NETINET6 NETUNIX NEWDB NIS
NISPLUS PIPELINING SCANF STARTTLS TCPWRAPPERS USERDB
USE_LDAP_INIT XDEBUG
 
============ SYSTEM IDENTITY (after readcf) ============
 (short domain name) $w = pluto
 (canonical domain name) $j = pluto.mydomain.com
(subdomain name) $m = pluto.mydomain.com
 (node name) $k = pluto
========================================================
 
Recipient names must be specified
$
 
The first line mentions the version of sendmail, followed by the “+Sun” string.
The latter indicates that this is a version of sendmail shipped with the Solaris OS or delivered through a Solaris patch.
Step 2 : Verify if the full path of the sendmail daemons running on the system corresponds to /usr/lib/sendmail
Step 2.1 – Verify the sendmail processes are running
For Solaris 9 and above :
# ps -ef |grep sendmail
root 516 1 0 Mar 19 ? 1:16 /usr/lib/sendmail -bd-q15m
smmsp 515 1 0 Mar 19 ? 0:04 /usr/lib/sendmail -Ac -q15m
# /usr/ucb/ps -auxwww | grep sendmail
root 6043 0.1 0.1 1280 968 console S 18:18:03 0:00 grep sendmail
root 516 0.0 0.2 8240 2648 ? S Mar 19 1:16 sendmail:accepting connections
smmsp 515 0.0 0.1 8240 2216 ? S Mar 19 0:03 sendmail:Queue runner@00:15:00 for /var/spool/clientmqueue
For Solaris 8 and below :
# ps -ef | grep send
root 271 1 0 Mar 10 ? 0:01 /usr/lib/sendmail -bd -q15m
 
# /usr/ucb/ps -auxwww | grep sendmail
root 271 0.0 0.3 4288 1512 ? S Mar 10 0:00 sendmail:accepting connections
Step 2.2 – Start the sendmail process
For Solaris 8:
#/etc/rc2.d/S88sendmail start
or
#/usr/lib/sendmail -bd -q15m
For Solaris 9:
#/etc/rc2.d/S88sendmail start
 
or
 
#/usr/lib/sendmail -bd -q15m
#/usr/lib/sendmail -Ac -q15m
For Solaris 10 and above :
 Below s10u9 ( update 9 ) :
 #svcadm enable svc:/network/smtp:sendmail
From s10u9 or 09/10 (Kernel Update 142909-17) :
 #svcadm enable svc:/network/smtp:sendmail
ps -ef|grep “sendmail -bd”
root 516 1 0 Mar 19 ? 1:16 /usr/lib/sendmail -bd
 
 #svcadm enable svc:/network/sendmail-client
ps -ef |grep “sendmail -Ac”
smmsp 515 1 0 Mar 19 ? 0:04 /usr/lib/sendmail -Ac -q15m
Step 3:  Verify is this is a Sun supported sendmail configuration.
If the full path of the sendmail daemons running on the system corresponds to /usr/lib/sendmail and the result of “step 1” indicates that this is a version of sendmail shipped with the Solaris OS or delivered through a Solaris patch, this a is a supportes Solaris sendmail configuration.

 

The Generic Procedure to Configure Sendmail for Receiving and Sending mails

Note:  Btw, this is just for guidelines and might vary depending on your environment

Step 1 – verify that the /etc/mail directory, sendmail.cf and submit.cf all exist.

The main sendmail daemon which listen on port 25 uses the sendmail.cf, while the secondary one uses the submit.cf.
# ls -ail /etc/mail 
total 924 
1445  drwxr-xr-x 3 root mail 1024 Mar 13 13:48 . 
1403  drwxr-xr-x 76 root sys 4608 Mar 27 11:59 .. 
19564 -rw-r–r– 1 root bin 163 Oct 29 12:40 Mail.rc 
3964  -rw-r–r– 1 root bin 1423 Oct 29 12:24 aliases 
2405  -rw-r—– 1 root smmsp 40960 Nov 6 14:45 aliases.db 
22855 drwxr-xr-x 9 root mail 512 Nov 6 14:35 cf 
21951 -rw-r–r– 1 root bin 5449 Dec 22 2006 helpfile 
4055  -rw-r–r– 1 root bin 9 Nov 30 10:43 local-host-names 
2977  -r–r–r– 1 root bin 39953 Dec 22 2006 local.cf 
1865  -rw-r–r– 1 root bin 1839 Oct 29 12:16 mailx.rc 
4048  lrwxrwxrwx 1 root root 11 Oct 29 12:24 main.cf -> sendmail.cf 
42821 -rw-r–r– 1 root root 50 Nov 30 11:09 relay-domains 
42667 -r–r–r– 1 root root 40551 Mar 14 14:58 sendmail.cf 
41752 -r–r–r– 1 root other 39900 Nov 6 14:35 sendmail.cf.old 
4331  -rw-r–r– 1 root root 40032 Nov 29 11:35 sendmail.cf_cust 
21801 -r–r–r– 1 root root 39875 Mar 13 13:48 sendmail.cf_orig 
4054  -r–r–r– 1 root bin 39895 Nov 28 10:39 sendmail.cf_save 
4049  lrwxrwxrwx 1 root root 8 Oct 29 12:24 sendmail.hf -> helpfile 
21832 -rw-r–r– 1 root bin 41448 Mar 14 15:01 submit.cf 
41761 -r–r–r– 1 root other 40241 Nov 6 14:35 submit.cf.old 
21818 -rw-r–r– 1 root root 40216 Mar 13 13:48 submit.cf_orig 
4056  -r–r–r– 1 root bin 40220 Nov 9 12:26 submit.cf_save 
4050  lrwxrwxrwx 1 root root 11 Oct 29 12:24 subsidiary.cf -> sendmail.cf 
42853 -rw-r–r– 1 root root 5 Nov 30 11:10 trusted-users 
4058  -rw-r–r– 1 root bin 0 Nov 30 10:47 trusted-users_save

Step 2 – Verify the sendmail.cf has the correct mailhost (this may differs from unique site, see summary) setup.

In Solaris 9 and above :

# grep DS sendmail.cf 
DS 
# grep Fallback sendmail.cf 
O FallbackSmartHost=mailhost$?m.$m$.
Basically the default behaviour for sendmail is mail exchanger will use dns servicces to query for the MX host entry of the domainname used in the addresses for forwarding the messages.
If this fails for some reason, eg. the DNS does not know the internet domains, it will fallback to use mailhost as defined in the  FallbackSmartHost.m4 macros:
define(`confFALLBACK_SMARTHOST’, `mailhost$?m.$m$.’)

In Solaris 8 and below :

 
# grep DS sendmail.cf 
DSmailhost$?m.$m$
 
sendmail by default is set to forward all messages to “mailhost” the smarthost. 
m4 macros:
 
define(`SMART_HOST’, `mailhost$?m.$m$.’)
Step 3 – Verify the submit.cf 
This is to route the local messages to the localhost port 25.
 
# grep MTAHost submit.cf 
D{MTAHost}[127.0.0.1]
# grep DS submit.cf 
DS

Step 4 – Verify that the port 25 is configured

In Solaris 10 with patch 142436-03 or higher:

Check the sendmail access status to remote clients from by the value of the local_only SMF property: 
svc:/network/smtp:sendmail/config/local_only = true 
A setting of true, as above, disallows remote access; false allows remote access. The default value is true. The following example shows the sequence of SMF commands used to enable sendmail to allow access to remote systems:
# svccfg -s svc:/network/smtp:sendmail setprop config/local_only = false 
# svcadm refresh svc:/network/smtp:sendmail 
# svcadm restart svc:/network/smtp:sendmail 

In Solaris 9 and above :

# grep Port sendmail.cf 
O DaemonPortOptions=Name=MTA-v4, Family=inet 
O DaemonPortOptions=Name=MTA-v6, Family=inet6 
O DaemonPortOptions=Port=587, Name=MSA, M=E
These are the default configuration which has port 25 setup by the above and port 587 has been setup to listen as well.

In Solaris 8 and below :

# grep Port sendmail.cf 
O DaemonPortOptions=Name=MTA-IPv4, Family=inet 
O DaemonPortOptions=Name=MTA-IPv6, Family=inet6 
O DaemonPortOptions=Port=587, Name=MSA, M=E 
m4 macros : DAEMON_OPTIONS(`NAME=MSA, Port=27, Addr=127.0.0.1, M=E’)
Step 5 – For receiving mail, ensure the local-host-names has populated with the localhost name:
# more local-host-names 
v4u-x1c
 

 

 

 

 

Steps to verify Logs for both inbound and outbound mail.

Step 1 –  Verify where syslogd(1M) will log sendmail syslog records to.

After a Solaris install you will find following entries in /etc/syslogd.conf:
*.err;kern.debug;daemon.notice;mail.crit        /var/adm/messages
mail.debug                      ifdef(`LOGHOST’, /var/log/syslog, @loghost)
 
and the purpose of these entries:
  •  syslog will log critical sendmail messages to /var/adm/messages
  •  if loghost can be resolved, syslogd will log sendmail syslog record to the /var/log/syslog file of that host.

Resolve the ‘loghost’ hostname:

$ getent hosts loghost
172.16.1.1      e450 loghost
$

Which syslog to check?

  • If loghost points to the same system where the sendmail service your want to troubleshoot is running on, you will find the sendmail syslog records in /var/log/syslog.
  • If loghost points to a different system, login into this system and verify the contents of /var/log/syslog.
  • If neither is the case, verify the contents of syslod.conf and verify to which host and/or file mail.debug records points to.

Step 2 –  Verify the contents of /var/adm/messages.

Syslog messages with priotity ‘mail.crit’ will by default logged to /var/adm/messages. As given in below example:
The following will be logged when the writing permissions to /var/spool/clientmqueue are not sufficient.
May 20 04:01:12 db7 sendmail[1872]: [ID 801593 mail.crit] NOQUEUE:SYSERR(oracle): can not write to queue directory /var/spool/clientmqueue/(RunAsGid=0, required=1): Permission denied
May 20 05:00:01 db7 sendmail[1961]: [ID 801593 mail.crit] NOQUEUE:SYSERR(sys): can not write to queue directory /var/spool/clientmqueue/(RunAsGid=0, required=1): Permission denied

Step 3 –  Verify the contents of the /var/log/syslog file or equivalent file.

Whenever the delivery status of a mail message changes, sendmail will record  this event via syslog.  As given in the example
Apr 16 10:58:56 mymailhost sendmail[24234]: [ID 801593 mail.info] m3G8wu8g024234: to=<joe.foe@extdomain.com>, ctladdr=<john@mymailhost.mydomain.com> (22960/117), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30920, relay=mailhost.extdomain.com. [172.17.1.4], dsn=2.0.0, stat=Sent (m3G8wug1009950 Message accepted for delivery)
 
Each line of information logged looks like this:
date host sendmail[pid]: qid: field1=value,field2=value,field3=value, …
  • The date is the month, day, and time that the line of information was logged.
  • The host is the name of the host that produced this information.
  • This can be different from the name of the host on which the logfiles are kept (see above).
  • The pid (process id) of the sendmail processes that produced the output.
  • The qid (queue identifier) that uniquely identifies each message on a given host.
  • The remainder is a list of fields which define values of who the sender or the recipient is  and whether delivery succeeded, failed, or was deferred.
The following is a list of some of these fields.
  • to=             The final recipient
  • from=           The envelope sender
  • ctladdr=        The controlling user
  • delay=          Total time to deliver
  • xdelay=         Transaction delay for this address only
  • mailer=         The delivery agent used
  • pri=            The initial priority
  • relay=          The host that sent or accepted the message
  • dsn=            The DSN status code
  • stat=           The status of delivery
  • size=           The size of the message
  • ntries=         The number of delivery attempts
With the above list you can identify which record points to a local delivery attempt,  a relay attempt, a forwarding attempt, etc…

Step 4 –  Verify the stat= field of this record.

This can be in the state queued when the recipient’s mailhost was not accessible.  After the same mailhost is accessible again, and after re-queuing of mail messages, the same message can be in the state Sent as mail was successful delivered.    In the case of bounced mail the stat= field will show the reason for the failure.

Step 5 – Verify the dsn= field of this record (when available).

When a mail server for some reason bounces a message, it may notify the envelope  sender of the problem with a DSN error code. The meaning of the DSN error codes are documented in RFC1893 (Enhanced Mail System Status Codes). As given in below example
Apr 11 10:25:29 e450 sendmail[16860]: [ID 801593 mail.info] m3B8PSVM016855: to=<user1@wrongdomain.org>, ctladdr=<user2@mydomain.org> (2031/2001), delay=00:00:01, xdelay=00:00:00, mailer=esmtp, pri=150549, relay=wrongdomain.org.be, dsn=5.1.2, stat=Host unknown (Name server: wrongdomain.org: host not found)
Here you can see that an attempt was made by user2@mydomain.org to send out a mail to a user “user1” on a non-resolvable domainname.
from the above log “dsn=5.1.2”  is can be decipherd as below ( according to RFC1893 )
5.X.X   Permanent Failure     :     A permanent failure is one which is not likely to be resolved by   resending the message in the current form.  Some change to the   message or the destination must be made for successful delivery.
 
 X.1.2   Bad destination system address    :     The destination system specified in the address does not  exist or is incapable of accepting mail.  For Internet mail  names, this means the address portion to the right of the   “@” is invalid for mail.  This codes is only useful for  permanent failures.

Step 6 –  Verify the reject= field of this record (when available).

When a mail message is processed by sendmail it will go through some check_rules used of Anti-spam control.
(see the  ANTI-SPAM CONFIGURATION CONTROL section of the /usr/lib/mail/README file).  If one of these rules rejects a mail message, sendmail will log this.
Apr 16 15:38:34 e450 sendmail[17707]: [ID 801593 mail.notice] m3GDcYW8017707: ru leset=check_mail, arg1=<nekkipfa1981@54sales.com>, relay=to1-84-91-48-146.netvis ao.pt [84.91.48.146], reject=553 5.1.8 <nekkipfa1981@54sales.com>… Domain of sender address nekkipfa1981@54sales.com does not exist
and
Apr 16 16:50:54 e450 sendmail[18581]: [ID 801593 mail.notice] m3GEorFQ018581: ru leset=check_rcpt, arg1=<a286e4184@yahoo.com.tw>, relay=203-73-236-153.adsl.dynam ic.seed.net.tw [203.73.236.153], reject=550 5.7.1 <a286e4184@yahoo.com.tw>…
 
In Solaris 9 and above, the sendmail releases uses two queues  
1.  /var/spool/mqueue
2. /var/spool/clientmqueue
Where as Solaris 8  uses only one i.e /var/spool/mqueue.
 The two queue  structure allows us not to give away setuid for root in the binary unlike S8 and below.

Steps to check the permissions for Mail Queues

Step 1 – Verify the /var/spool/mqueue exist, also has the following permissions and ownerships

For S9 and above :
# ls -ail /var/spool/mqueue
drwxr-x—   2 root     bin         /var/spool/mqueue
#/usr/bin/chmod 750 /var/spool/mqueue
#/usr/bin/chown root:bin /var/spool/mqueue
For S8 and below :
# ls -ail /var/spool/mqueue
drwxr-x—   2 root     bin         /var/spool/mqueue
#/usr/bin/chmod 750 /var/spool/mqueue
#/usr/bin/chown root:bin  /var/spool/mqueue

Step 2 – For S9 and above, verify the /var/spool/clientmqueue exist, also has the following permission and ownership

# ls -ail /var/spool/clientmqueue
drwxrwx—   2 smmsp    smmsp               /var/spool/clientmqueue
#/usr/bin/chmod 770 /var/spool/clientmqueue
#/usr/bin/chown smmsp:smmsp  /var/spool/client/mqueue
 
We can also use the -d44.5 switch on sendmail to debug permission problems:
# /usr/lib/sendmail -v -d44.5 someone@somewhere.com < /etc/hosts
 

Steps to Validate DNS client configuration  (and to verify Mail exchanger lookups are successful (if using DNS for mail exchanger lookups).

Note:  Depending on you Infrastructure setup, some of the the resolution steps needs to be implemented by DNS Administrators

Is system configured as a DNS client:

Solaris 8, 9 and 10: 
Check the file “/etc/nsswitch.conf ” for the  “dns” keyword in   “hosts:” line.
Check the file /etc/resolv.conf exists, to make sure that you have valid nameserver IP addresses
To check for a mail exchanger (MX) record for a domain:
Using nslookup
# nslookup (type this command and return)
Default Server: nameserver.somedomain.COM
Address: 129.168.1.2
> set type=MX (type this command and return)
> yahoo.com (type in the domain in question, results follow)
Server: nameserver.somedomain.COM
Address: 129.168.1.2
 
Non-authoritative answer: yahoo.com preference = 5, mail exchanger = mx1.yahoo.com
yahoo.com preference = 5, mail exchanger = mx2.yahoo.com
yahoo.com preference = 5, mail exchanger = mx3.yahoo.com
yahoo.com preference = 5, mail exchanger = mx4.yahoo.com
 
Authoritative answers can be found from:
yahoo.com nameserver = ns1.yh.net
yahoo.com nameserver = ns2.yh.net
yahoo.com nameserver = ns3.yh.net
yahoo.com nameserver = ns4.yh.net
yahoo.com nameserver = ns5.yh.net
mx1.yahoo.com internet address = 65.54.244.8
mx1.yahoo.com internet address = 65.54.245.8
mx1.yahoo.com internet address = 65.54.244.136
mx2.yahoo.com internet address = 65.54.244.168
mx2.yahoo.com internet address = 65.54.244.40
mx2.yahoo.com internet address = 65.54.245.40
mx3.yahoo.com internet address = 65.54.244.200
mx3.yahoo.com internet address = 65.54.244.72
mx3.yahoo.com internet address = 65.54.245.72
mx4.yahoo.com internet address = 65.54.245.104
mx4.yahoo.com internet address = 65.54.244.104
mx4.yahoo.com internet address = 65.54.244.232
ns1.yh.net internet address = 207.68.160.190
ns2.yh.net internet address = 65.54.240.126
ns3.yh.net internet address = 213.199.161.77
ns4.yh.net internet address = 207.46.66.126
ns5.yh.net internet address = 65.55.238.126
 
Using dig
 
# dig -t mx yahoo.com
 
; <<>> DiG 8.3 <<>> -t yahoo.com
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 5, ADDITIONAL: 17
;; QUERY SECTION:
 
;; yahoo.com, type = MX, class = IN
 
;; ANSWER SECTION:
 
yahoo.com. 47m34s IN MX 5 mx2.yahoo.com.
yahoo.com. 47m34s IN MX 5 mx3.yahoo.com.
yahoo.com. 47m34s IN MX 5 mx4.yahoo.com.
yahoo.com. 47m34s IN MX 5 mx1.yahoo.com.
 
;; AUTHORITY SECTION:
 
yahoo.com. 6h4m35s IN NS ns1.yh.net.
yahoo.com. 6h4m35s IN NS ns2.yh.net.
yahoo.com. 6h4m35s IN NS ns3.yh.net.
yahoo.com. 6h4m35s IN NS ns4.yh.net.
yahoo.com. 6h4m35s IN NS ns5.yh.net.
 
;; ADDITIONAL SECTION:
 
mx2.yahoo.com. 47m34s IN A 65.54.244.168
mx2.yahoo.com. 47m34s IN A 65.54.244.40
mx2.yahoo.com. 47m34s IN A 65.54.245.40
mx3.yahoo.com. 47m34s IN A 65.54.244.200
mx3.yahoo.com. 47m34s IN A 65.54.244.72
mx3.yahoo.com. 47m34s IN A 65.54.245.72
mx4.yahoo.com. 47m34s IN A 65.54.245.104
mx4.yahoo.com. 47m34s IN A 65.54.244.104
mx4.yahoo.com. 47m34s IN A 65.54.244.232
mx1.yahoo.com. 47m34s IN A 65.54.244.8
mx1.yahoo.com. 47m34s IN A 65.54.245.8
mx1.yahoo.com. 47m34s IN A 65.54.244.136
ns1.yh.net. 1d1h41m34s IN A 207.68.160.190
ns2.yh.net. 7m13s IN A 65.54.240.126
ns3.yh.net. 7m13s IN A 213.199.161.77
ns4.yh.net. 7m13s IN A 207.46.66.126
ns5.yh.net. 7m13s IN A 65.55.238.126
 
;; Total query time: 62 msec
 
;; FROM: solarishost to SERVER: default — 129.168.1.2
;; WHEN: Thu Apr 3 15:34:41 2008
;; MSG SIZE sent: 29 rcvd: 479<
 
If an MX record is not found:
Doublecheck with DNS adminstrator if one should exist and if so, query it directly using the hostname and address supplied by your DNS admin and use the command:
  • nslookup <hostname or IP address>
  •  dig <hostname or IP address>
  •  getent hosts <hostname or IP address>

Using getent

root# getent hosts mailhost
129.148.9.192 mailhost.foo.com
129.148.13.5 mailhost.foo.com
if getent cannot find a match for mailhost it will return the prompt without output:
 
root# getent hosts mailhost
root#
Using /usr/lib/sendmail -d
root# /usr/lib/sendmail -d0.11 -bp
 
Version 8.13.8+Sun
 
Compiled with: DNSMAP LDAPMAP LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8 MIME8TO7 NAMED_BIND NDBM NETINET NETINET6 NETUNIX NEWDB NIS NISPLUS PIPELINING SCANF STARTTLS TCPWRAPPERS USERDB USE_LDAP_INIT XDEBUG
 
OS Defines: HASCLOSEFROM HASFCHOWN HASFCHMOD HASFDWALK HASGETUSERSHELL HASINITGROUPS HASLDAPGETALIASBYNAME HASLSTAT HASNICE HASRANDOM HASRRESVPORT HASSETREGID HASSETREUID HASSETRLIMIT HASSETSID HASSETVBUF HASURANDOMDEV HASSTRERROR HASULIMIT HASUNAME HASUNSETENV HASWAITPID IDENTPROTO IP_SRCROUTE SAFENFSPATHCONF SYS5SETPGRP SYSTEM5 USE_DOUBLE_FORK USE_SA_SIGACTION USE_SIGLONGJMP USESETEUID
 
Kernel symbols: /dev/ksyms
Conf file: /etc/mail/submit.cf (default for MSP)
Conf file: /etc/mail/sendmail.cf (default for MTA)
Pid file: /var/run/sendmail.pid (default)
Canonical name: solarishost.nisplus.com
UUCP nodename: solarishost
a.k.a.: solarishost.nisplus.com
a.k.a.: [10.10.11.88]
a.k.a.: [127.0.0.1]
a.k.a.: loghost
Conf file: /etc/mail/sendmail.cf (selected)
Pid file: /var/run/sendmail.pid (selected)
 
============ SYSTEM IDENTITY (after readcf) ============
 
(short domain name) $w = solarishost
(canonical domain name) $j = solarishost.nisplus.com
(subdomain name) $m = nisplus.com
(node name) $k = solarishost
 
========================================================
 
/var/spool/mqueue is empty
Total requests: 0
 
Using /usr/lib/sendmail -bt test mode:
 
To query a domain for mailserver (MX) records:
 
root# /usr/lib/sendmail -bt
 
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
 
Enter <ruleset> <address>
 
> /mx yahoo.com [type in /mx domain2query.com and return]
 
getmxrr(yahoo.com) returns 7 value(s):
 
c.mx.mail.yahoo.com.
f.mx.mail.yahoo.com.
b.mx.mail.yahoo.com.
a.mx.mail.yahoo.com.
e.mx.mail.yahoo.com.
d.mx.mail.yahoo.com.
g.mx.mail.yahoo.com.
 
(control-D to exit)

 

Sendmail Troubleshooting for Mail Receive Issues

 

 

First refer the Section “Sendmail Services” from Mail Sending Troubleshooting above.

In addition to the above checks,  check sendmail “secure by default” status

 

Below procedure helps you to determine if your sendmail configuration is using the “Secure by Default” profile. This might be a possible cause for the symptom “can’t receive mail”. The “Secure by Default Network Profile” is an installation enhancement that arrived with Solaris 10U3
To determine if the system is configured for Secure by Default:

# cd /var/svc/profile

ls -l gen* lrwxrwxrwx 1
root root 25 Mar 26 06:13 generic.xml -> ./generic_limited_net.xml
-r–r–r– 1 root sys 11300 Dec 21 2006 generic_limited_net.xml
-r–r–r– 1 root sys 5592 Dec 21 2006 generic_open.xml

# ps -ef | grep sendmail
smmsp 10561 10559 0 11:57:52 ? 0:00 /usr/lib/sendmail -Ac -q15m
smmsp 10559 1 0 11:57:52 ? 0:00  /usr/lib/sendmail -Ac -q15m root
10560 1 0 11:57:52 ? 0:00 /usr/lib/sendmail -bd -q15m – C /etc/mail/local.cf

           .. Last line points to “local.cf”

To determine if this property is set for sendmail:

# svccfg -s sendmail listprop | grep local_only
config/local_only boolean true

   … But default this setting is  false

 

To re-enable all network services including sendmail:

# netservices open

 

To re-enable the sendmail network service only:

Determine if this property is set for sendmail:

# svccfg -s sendmail listprop | grep local_only config/local_only boolean true <— The default is false

Disable sendmail:

# svcadm -v disable sendmail

Turn local_only mode back to false: 

# svccfg -s sendmail setprop config/local_only=false

Verify the change:

# svccfg -s sendmail listprop | grep local_only
config/local_only boolean false

Refresh the sendmail service so that the change takes effect:

# svcadm -v refresh sendmail

… About command will refresh svc:/network/smtp:sendmail.

Enable sendmail again:

# svcadm -v enable sendmail svc:/network/smtp:sendmail enabled.

Verify the change was made to sendmail only:

# pwd
/var/svc/profile

# ls -l gen* lrwxrwxrwx 1 root root 25 Mar 26 06:36 generic.xml -> ./generic_limited_net.xml
-r–r–r– 1 root sys 11300 Dec 21 2006 generic_limited_net.xml
-r–r–r– 1 root sys 5592 Dec 21 2006 generic_open.xml

        … sendmail now running in a default state again which will look at the sendmail.cf and not the local.cf

# ps -ef | grep sendmail

smmsp 2411 1 0 06:50:51 ? 0:00 /usr/lib/sendmail -Ac -q15m
root 2412 1 0 06:50:51 ? 0:00 /usr/lib/sendmail -bd -q15m

 

 

 

 

 

 

 

 

 

 

Refer the “Network Connectivity” Section from the “Troubleshooting Mail Sending issues” mentioned above

 

 

 

 

 

Refer the “DNS Checks” Section from the “Troubleshooting Mail Sending issues” mentioned above

 

 

 

 

 

 

User account details can be either located in the local passwd file or other name services, depending on the site configurations.
The steps presented here should work regardless of such configurations. In order to verify the existence of a user account, login into the system, run the id command with the username as the argument. For example, to confirm that the username postgres exists, run for the following command:
 
$ id postgres
uid=90(postgres) gid=90(postgres)
 
If the user does not exist, it will report invalid user name:
$ id mysql
id: invalid user name: “mysql”
It may be also useful to retrieve the entire passwd entry for the user account using the getent command. This will provide details such as user real name to confirm this account actually belongs to the user who reported the issue.
$ getent passwd postgres
postgres:x:90:90:PostgreSQL Reserved UID:/:/usr/bin/pfksh
 
If the user account does not exist, it will return back to the shell prompt:
$ getent passwd mysql
$
At this point, if the user account cannot be located, you may want to consider re-creating the account for the user.

 

 

 

 

The sendmail daemon (/usr/lib/sendmail -bd -q15m) listens on port 25 and when a message is received, it is determined to be destined for the local mail box or externally after rules rewrite. For local mail box delivery, the message is routed to the mailer local (Mlocal) in the sendmail.cf which is termed the mail.local binary.

Step 1. Ensure the mail.local binary exists:

 

# ls -ail /usr/lib/mail.local
3193 -r-xr-xr-x 1 root bin 63104 Apr 5 2007 /usr/lib/mail.local

 

Step 2. Set ownership and mode on the mail.local binary

 

#chown root:bin /usr/lib/mail.local
#chmod 555 /usr/lib/mail.local

 

Step 3. Ensure the sendmail.cf has the defaullt entry for the mailer “local” :

Mlocal, P=/usr/lib/mail.local, F=lsDFMAw5:/|@qPSXmnz9, S=EnvFromSMTP/HdrFromL, R=EnvToL/HdrToL,
T=DNS/RFC822/SMTP,
A=mail.local -l

 

 

The Mail should always be delivered to a local file system so that the user agent can pull mail from the mail spool and store it readily in the local mailbox. Do not use

Note :    NFS-mounted file systems as the destination for a user’s mailbox. Specifically, do not direct mail to a mail client that is mounting the /var/mail file system from a remote server. Mail for the user, in this instance, should be addressed to the mail server and not to the client host name. NFS-mounted file systems can cause problems with mail delivery and handling.

The /etc/mail/aliases file and name services such as NIS, NIS+ and LDAP provide mechanisms for creating aliases for electronic mail addresses. So, users do not need to know the precise local name of a user’s mailbox.

Below procedure helps you to verify the permissions of /var/mail and its NFS mount state when using remote access.

# ls -ld /var/mail
drwxrwxrwt 3 root mail 25088 May 15 2007 /var/mail
#

…..By Default Sticky Bit should set for this directory

 

By default, sendmail (All versions) will refuse mail connections when load average rises above 12. Load average can be seen with the unix command “/usr/bin/uptime”.

# uptime
11:32am up 1 day(s), 23:13, 1 user, load average: 0.07, 0.08, 0.09

Where:

0.07 = 1 minute
0.08 = 5 minutes
0.09 = 15 minutes

This can be a problem with multiprocessor machines where load averages.High Load average ceases sendmail to stop forking processes.

Below Procedure Helps to set the load average to higher limit(RefuseLA):

In sendmail 8.8.8(or later) in sendmail.cf
#cd /usr/lib/mail/cf
#vi sendmail.mc
divert(-1)
divert(0)dnl
VERSIONID(`@(#)main-v7sun.mc 1.5 (Sun) 09/12/01′)
OSTYPE(`solaris8′)dnl
DOMAIN(`solaris-generic’)dnl
MAILER(`local’)dnl
MAILER(`smtp’)dnl
define(`SMART_HOST’, `mailhost$?m.$m$.’)dnl
define(`confREFUSE_LA’,21)dnl$
/usr/ccs/bin/m4 ../m4/cf.m4 sendmail.mc > sendmail.cf
One can use the -d3.30 to see if the load is really that high:
# /usr/lib/sendmail -d3.30 some.body@unixadminschool.com < /etc/passwd
getla: 4
getla: 4
getla: 4
shouldqueue: CurrentLA=4, pri=30638: false (CurrentLA < QueueLA)

Note

  • In Solaris 8, this issue only affect inbound connection, however in Solaris 9 this affects also the outbound messages that is send locally.
  • This is due to the fact that in Solaris 9/10 there are two queues in sendmail. The local message has to connect back to the 127.0.0.1:25 which is then serviced by the main sendmail daemon which in turn send this out.

Aliasing is the process of converting one recipient name into another.  It is essential for sendmail to be able to expand the mail address in the aliases db correctly before the mailer mail.local is called. Otherwise, messages will not be delivered successfully to the local mail box.

Step 1. Ensure the default /etc/mail/aliases file exists and set it’s permisions :

# ls -ail /etc/mail/aliases
3162 -rw-r–r– 1 root bin 1423 Jul 14 2007 /etc/mail/aliases
#chmod 644 /etc/mail/aliases
#chown root:bin /etc/mail/aliases

 

Step 2. Ensure the AliasFile option is set to the correct aliases file :

/etc/mail/sendmail.cf :

O AliasFile=/etc/mail/aliases

Or m4 macros to use :

define(`ALIAS_FILE’, `/etc/mail/aliases’)

Step 3. When adding a new entry in the aliases file, ensure this is updated into the aliases.db file :

#vi /etc/mail/aliases :
me: stiffer, marie@[129.123.456.78]

#/usr/sbin/newaliases
/etc/mail/aliases: 13 aliases, longest 31 bytes, 171 bytes total

 

Step 4. Check The name service switch

The Names Service Switch Configuration file , i.e  nsswitch.conf ,  has a database for aliases  , so ensure the /etc/nsswitch.conf has the correct setting for the aliases database that you are using or updated

Sample entry in “/etc/nsswitch.conf “:
aliases: files nisplus ldap

Step 5. Ensure the user address is seen or expandable by sendmail :

#/usr/lib/sendmail -bt
> /map aliases toor
map_lookup: aliases (toor) returns root (0)
> /map aliases me
map_lookup: aliases (me) returns stiffer, marie@[129.123.456.78] (0)

Or use debug switch 37.88-91 :

#/usr/lib/sendmail -v -d37.88-91 me < /etc/hosts
.
.
Look for the below :

>>> MAIL From:<somebody@aserver.unixadminschool.com> SIZE=676
250 2.1.0 <somebody@aserver.unixadminschool.com>… Sender ok
>>> RCPT To:<me@aserver.unixadminschool.com>
>>> DATA
050 <me@aserver.unixadminschool.com>… aliased to stiffer, marie@[129.123.456.78]

 

Ensure the /etc/mail/local-host-names has the local hostname of the machine receiving the message.

# cat /etc/mail/local-host-names | grep rcvhost
rcvhost

where rcvhost is the machine name.

 

 

Refer the “SysLog Checks” Section from the “Troubleshooting Mail Sending issues” mentioned above

 

 

 

 

 

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

1 Response

  1. September 16, 2015

    […] Read – Troubleshooting Reference […]

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us