Solaris Troubleshooting Sendmail: Investigating Sendmail Errors from Syslog

Step 1 –  Verify where syslogd will log sendmail syslog records to.

After a Solaris install you will find following entries in /etc/syslogd.conf:


mail.debug                      ifdef(`LOGHOST’, /var/log/syslog, @loghost)


*.err;kern.debug;daemon.notice;mail.crit        /var/adm/messages

This has the effect that:
– syslog will log critical sendmail messages to /var/adm/messages
–  if loghost can be resolved, syslogd will log sendmail syslog record to the /var/log/syslog file of that host.

Resolve the ‘loghost’ hostname:
$ getent hosts loghost
172.16.1.1      e450 loghost
$

If loghost points to the same system where the sendmail service your want to troubleshoot is running on, you will find the sendmail syslog records in /var/log/syslog.
If loghost points to a different system, login into this system and verify the contents of /var/log/syslog. If neither is the case, verify the contents of syslod.conf and verify to which host and/or file mail.debug records points to.

Step 2 –  Verify the contents of /var/adm/messages.

Syslog messages with priotity ‘mail.crit’ will by default logged to /var/adm/messages.

example:

The following will be logged when the writing permissions to /var/spool/clientmqueue are not sufficient.

...
May 20 04:01:12 db7 sendmail[1872]: [ID 801593 mail.crit] NOQUEUE:SYSERR(oracle): can not write to queue directory /var/spool/clientmqueue/(RunAsGid=0, required=1): Permission denied
May 20 05:00:01 db7 sendmail[1961]: [ID 801593 mail.crit] NOQUEUE:SYSERR(sys): can not write to queue directory /var/spool/clientmqueue/(RunAsGid=0, required=1): Permission denied
...

Step 3 –  Verify the contents of the /var/log/syslog file or equivalent file.

Whenever the delivery status of a mail message changes, sendmail will record this event via syslog.

Example:

Apr 16 10:58:56 mymailhost sendmail[24234]: [ID 801593 mail.info] m3G8wu8g024234: to=<joe.foe@extdomain.com>, ctladdr=<john@mymailhost.mydomain.com> (22960/117), delay=00:00:00,
xdelay=00:00:00, mailer=relay, pri=30920, relay=mailhost.extdomain.com. [172.17.1.4], dsn=2.0.0, stat=Sent (m3G8wug1009950 Message accepted for delivery)

Each line of information logged looks like this:

date host sendmail[pid]: qid: field1=value,field2=value,field3=value, …

– The date is the month, day, and time that the line of information was logged.
– The host is the name of the host that produced this information.
  This can be different from the name of the host on which the logfiles are kept (see above).
– The pid (process id) of the sendmail processes that produced the output.
– The qid (queue identifier) that uniquely identifies each message on a given host.
– The remainder is a list of fields which define values of who the sender or the recipient is
  and whether delivery succeeded, failed, or was deferred.

The following is a list of some of these fields.

to=             The final recipient
from=           The envelope sender
ctladdr=        The controlling user
delay=          Total time to deliver
xdelay=         Transaction delay for this address only
mailer=         The delivery agent used
pri=            The initial priority
relay=          The host that sent or accepted the message
dsn=            The DSN status code     
stat=           The status of delivery
size=           The size of the message
ntries=         The number of delivery attempts

With the above list you can identify which record(s) points to a local delivery attempt,
a relay attempt, a forwarding attempt, etc…


Step 4 –  Verify the stat= field of this record.

This can be in the state queued when the recipient’s mailhost was not accessible. After the same mailhost is accessible again, and after re-queuing of mail messages,
the same message can be in the state Sent as mail was successful delivered. In the case of bounced mail the stat= field will show the reason for the failure.

Step 5 – Verify the dsn= field of this record (when available).

When a mail server for some reason bounces a message, it may notify the envelope sender of the problem with a DSN error code.
The meaning of the DSN error codes are documented in RFC1893 (Enhanced Mail System Status Codes).
example:

Apr 11 10:25:29 e450 sendmail[16860]: [ID 801593 mail.info] m3B8PSVM016855: to=<joe@wrongdomain.org>, ctladdr=<john@mydomain.org> (2031/2001), delay=00:00:01, xdelay=00:00:00, ma
iler=esmtp, pri=150549, relay=wrongdomain.org.be, dsn=5.1.2, stat=Host unknown (Name server: wrongdomain.org: host not found)

Here you can see that an attempt was made by john@mydomain.org to send out a mail to a user
joe on a non-resolvable domainname.

dsn=5.1.2 is decipherd according to RFC1893

    5.X.X   Permanent Failure

       A permanent failure is one which is not likely to be resolved by  resending the message in the current form.  Some change to the   message or the destination must be made for successful delivery.

       X.1.2   Bad destination system address

          The destination system specified in the address does not   exist or is incapable of accepting mail.  For Internet mail  names, this means the address portion to the right of the      “@” is invalid for mail.  This codes is only useful for  permanent failures.

Step 6 –  Verify the reject= field of this record (when available).

When a mail message is processed by sendmail it will go through some check_rules used of Anti-spam control.  (see the  ANTI-SPAM CONFIGURATION CONTROL section of the /usr/lib/mail/README file)  If one of these rules rejects a mail message, sendmail will log this.
Example:

Apr 16 15:38:34 e450 sendmail[17707]: [ID 801593 mail.notice] m3GDcYW8017707: ru leset=check_mail, arg1=<nekkipfa1981@54sales.com>, relay=to1-84-91-48-146.netvis ao.pt [84.91.48.
146], reject=553 5.1.8 <nekkipfa1981@54sales.com>… Domain of sender address nekkipfa1981@54sales.com does not exist

and

Apr 16 16:50:54 e450 sendmail[18581]: [ID 801593 mail.notice] m3GEorFQ018581: ru leset=check_rcpt, arg1=<a286e4184@yahoo.com.tw>, relay=203-73-236-153.adsl.dynam ic.seed.net.tw [
203.73.236.153], reject=550 5.7.1 <a286e4184@yahoo.com.tw>… Relayin Solaris 9 Operating System
Solaris 8 Operating System


Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

4 Responses

  1. Babaji says:

    Thanks Ram . Nice article.
    Can you post more on SMTP configurations and troubleshooting techniques.

  2. Ramdev Ramdev says:

    @Babaji –
    Sure, I am adding it to writing Queue

  3. manisha sharma says:

    Thank You Ram, learned about it .Yesterday faced issue where we are getting system undeliverable message when run mailx with -r option .It was working for the user until day before yestrday but stopped working.Any idea that why such error comes. Exchange team says that they have no idea about .

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us