Blogs

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that have been used in the blog.
  • Login
    Login Login form

Audit Logs on Cisco Unified Communications Manager (CUCM)

Posted by on in Unified Communications
  • Font size: Larger Smaller
  • Hits: 29790
  • 9 Comments
  • Print

Ever since I started working with Cisco Call Manager (circa 1999 - yikes) I have always wanted a way to implement an Authentication/Authorization/Accounting model.  Now,  I finally got everything I wanted.   The Authentication/Authorization need was addressed somewhere around CM 3.1.4b with a web application add-on called Multi-Level Access (MLA).  Since the initial release, MLA has been incorporated into the CM software (starting around CM 4.1.3).  With 5x/6x/7x the MLA term has gone by the wayside but the feature is alive and well (and quite robust).  Now, with CUCM 7.1(2) we have a decent Accounting or Auditing solution available.

Background

I will keep the background discussion to a minimum, because I am sure most readers of this blog understand the need for an auditing solution.  To summarize, admins need a way to look at an audit trail for administrative changes that occur in their CUCM cluster.  Basically, in large organizations there is a need to look at "what has changed" so that if there is an issue reported that is related to human error, the error can be quickly identified and fixed.  There is also an accountability aspect as it is likely that senior support folks want to identify education gaps in their lower tier support teams.

Software Revisions

The ability to enable auditing for administrative tasks was included in CUCM version 7.1(2) and later.  In my lab, I tested this feature using a CUCM 7.1(3b)SU2 cluster.

Enabling Audit Logs

By default, audit logs for administrative level tasks are enabled.  To access this feature, you will need to browse the audit log parameters from Cisco Unified Serviceability.  The following procedures can be used to accomplish this task:

1. Connect to Cisco Unified Serviceability on your CUCM cluster: https://mypublisher/ccmservice/

2. Go to Tools>Audit Log Configuration

3. Depending on your particular operational model, you may have people with different levels of authorization.  Authorization levels are implemented via the Groups and Roles configuration objects.  If you have a need to customize roles or Group/Role assignments you will want to look at the "Standard Audit Log Administration" role to understand the level of access required for users/groups that need to look at audit logs.

Accessing Audit Logs

At this point, you have enabled audit logs for administrative tasks.  As with other logs and trace files on the system, you can access these logs in several ways.  For this example, we will walk through using RTMT to view audit logs.

1. Load RTMT on your admin workstation

2. Connect to your CUCM cluster using a user ID that has the "Standard Audit Log Administration" role (or equivalent) assigned.

3. Go to System>Tools>Trace>Trace and Logs Central

4. Under "Trace and Logs Central", you will see an "Audit Logs" folder.  Double click the folder to start the log collection wizard.

One of the benefits in using RTMT is that you can collect and download AuditLogs either as a one time request or you can schedule the download.  For those that like to run pro-active analysis (via scripts and the like) you may want to look at the "Schedule Download" option.  As with other traces/logs, once you specify the search criteria RTMT will provide a folder tree structure that can be used to navigate to the audit logs stored on the server.

 

Deciphering the Audit Log

The log file entries are lengthy but parsing them is not at all difficult.  Let's look at an example.  I have a dummy phone on my CUCM cluster.  The phone was already created prior to testing the audit log feature.  What I did was add a phone number to the device.  In the AuditLog this action is captured as follows:

04/06/2010 11:03:15.295 |LogMessage
UserID :ccmadministrator
ClientAddress :192.168.1.4
Severity :5
EventType :GeneralConfigurationUpdate
ResourceAccessed:CUCMAdmin

EventStatus :Success
AuditDetails : record in table numplan with key field dnorpattern = 2025552222 added
ComponentID :Cisco CUCM Administration
App ID:Cisco Tomcat
Cluster ID: Node ID:iecucm01

Now, the log entry itself will be a single line in the trace file but I have broken it out in something a tad more legible.  You will see that the log identifies the user who performed the change (UserID), the IP address of the machine the user was using (ClientAddress) along with other pertinent information.  Of particular interest is the AuditDetails section of the logged event.  In this example, we can see that I added a directory number.   Before dissecting this section, we should address an obvious question: which device was this performed on?

This is where things are tricky.  If you look at previous log entries, you will find when the user (ccmadministrator in our example) connected to the CCMadmin web portal.  However, the AuditLog will not track this user's movements through the various web pages.  So, if I clicked on a device, and then clicked on "Add new line".  Those events are not recorded.  Only when I actually "save" the change will it be recorded in the AuditLog.

Why?  Well, it boils down to the fact that the Audit Log is tracking changes to the database.  It's focus is on what has changed in the database, not what has changed in the admin interface.  This is actually a necessary distinction, since changes to the database can come from a variety of sources:  BAT, CCMAdmin, AXL/SOAP (e.g. UCCX creates a JTAPI trigger).  If you look at our example above, you will see that the AuditDetails says:

record in table numplan with key field dnorpattern=2025552222 added

Numplan is a database table and if we wanted to find out what device this number was added to, we would need to take a few extra steps.  One step could be to run an ad hoc query from the command line to see what gives:

admin:run sql select d.name as device,n.dnorpattern,dmap.numplanindex
from numplan as n inner join devicenumplanmap as dmap on dmap.fknumplan=n.pkid
inner join device as d on dmap.fkdevice=d.pkid
where n.dnorpattern='2025552222'
device dnorpattern numplanindex
=============== =========== ============
SEP00258418EE10 2025552222 3

From this output, we have identified the device and the line appearance the new number is assigned.  Yeah, I know the query is ugly.  I can't resist you see, I do most of my admin tasks via custom queries.  That's just me.  You could certainly figure this out via the CCMAdmin web pages by going to Call Routing>Route Plan Report.  Then search for the directory number "2025552222".

So, you still have some work to do when you want to analyze the audit log, but the data now available is a great improvement over what you would have to do in the past.  I won't even waste your cycles on that discussion.

Other Things That Are Logged

As you play around with the Audit Log, you will find that it captures other interesting things as well.  Such as commands entered by users on the CLI.  This is handy if you want to find out who restarted the TomCat service or who reloaded the server.  Small things like that.  Of course, if you want that data to have any meaning, you may want to investigate the CLI command: set account name.

The Audit Log will also capture events like a user uploading a custom banner page to the CUCM system.  The custom banner page is another new security feature added to CUCM 7.1(2).

Where is the Audit Log

For those folks who don't dig on using RTMT for certain tasks, and would just rather download the file or view it directly on the console I have two pieces of information:

1. The files are stored here:  activelog audit/AuditApp/*

2. If you go this route, I recommend downloading the files to your workstations and then viewing/parsing/filtering.  Especially during production hours.

Conclusion

Well, the AuditLog feature isn't 100% perfect as you still have to tear yourself away from CNN or ESPN to actually do some work.  But, it is getting there and the information that is recorded in the AuditLog is very handy when you need to determine the "how", "when", and "why".  Hmmm, maybe it won't tell you the "why" but it sure will point you to the person that can!

Comments

  • Guest
    Steve Thursday, 08 April 2010

    Thanks for Log Guidance

    Appreciate your detailed and insightful tutorial. It's very timely because I've been searching on and off for the last few weeks for a method of auditing changes to a 6.0(2) cluster. I was wondering if there may be a way of tracking or capturing tomcat login and page serves for a specific user account. Can you think of a trace or capture that might accomplish this?
    Humbly,
    Steve Sutton

  • William Bell
    William Bell Wednesday, 14 April 2010

    Re: Thanks for Log Guidance

    Steve,

    Sorry for the delay in getting back to you. I missed this comment in my e-mail stream. The audit logs outlines in the article can be used for this purpose. Back in the day, I would use the IIS logs (pre-appliance model) to do something similar but that was a chore. You can see similar data with tomcat. On the CUCM system (usually the publisher node) you can view a daily access record for the tomcat service:

    admin:file list activelog tomcat/logs/localhost* detail

    You can then view one of these files to see if it is close to what you need. I am not sure if there is any more value in tomcat that the audio logs don't already provide but you can determine what best meets your requirements. (NOTE: You can also browse the tomcat logs using the same methods described in this article).

    Now, if you are looking for a way to pull or push the traces off box and then post process using perl or something similar, then you have a few hoops to jump through. You can push tomcat traces to a syslog server. Take a look at the CUCM Serviceability configurations (Alarm configurations). Tomcat traces can be configured under the platform services for your server.

    I am not sure if there is a way to push auditlogs to a syslog server but you can pull them on a scheduled basis using RTMT. There are other methods, but using RTMT is pretty straight forward.

    Regards,
    Bill

  • Guest
    Guy Kelly Thursday, 16 September 2010

    Audit Logs

    Hi William,

    Actually this article is incorrect. The logs you are enabling above are the database logs and are filed under the informixauditlogs on RTMT. Application audit logs are enabled by default and are the checkboxes listed in the Application Audit Log Settings above.

    Cheers
    Guy Kelly

  • William Bell
    William Bell Wednesday, 27 October 2010

    Re: Audit Logs

    Guy,

    Yup, you are absolutely correct. I have nothing to say except thanks. I have made the correction and plan to come back and expand upon blog later.

    Thanks,
    Bill

  • Guest
    Vladimir Saturday, 12 May 2012

    Delegate Permissions

    Hi!
    Thank you for this very usefull article!
    Is there any way to delegate to any user a permission to manage only some pool of phones?
    In large organisation with many Regions anf offices we have to give to local admins roles:
    Standard CCM end users
    Standard CCM phone management
    Standard CCM user management
    Every thing works fine, local admin can add and change phones in his office.
    The problem that ye can can change and delete ANY phone in Organization.

    Is there any way to resolve it?
    Thanks
    Vladimir

  • William Bell
    William Bell Saturday, 12 May 2012

    Re: Delegate Permissions

    Vladimir,

    The behavior you are observing is by design. The CUCM authorization model only provides for what web pages you can access and the permission level on those pages. There is no way you can provide for a mult-tenant admin model using the native CUCM software.

    There are third-party applications that address this need. I haven't looked at Cisco's Unified Provisioning Manager, but it may provide what you are looking for (and then some). You could also look at Unimax, 2Ring, and similar partners. I cannot recommend a solution on this site. Perhaps query: "cucm multi-tenant provisioning" or "cucm multi-tenant administration" in Google.

    HTH.

    Regards,
    Bill

  • Guest
    ahmed Wednesday, 12 September 2012

    Good Response !

    commenting on response of :William Bell, February 24, 2011 ,

    Really this is good info ,,,pls include some examples sql commands ,
    i appreiciate for good work :)
    :)

  • William Bell
    William Bell Wednesday, 12 September 2012

    Re: Good Response

    Ahmed,

    I have a series that goes through various SQL queries. This series is hosted on my personal blog: http://ucguerrilla.com.

    HTH.

    -Bill (@ucguerrilla)

  • Guest
    Sally Friday, 07 February 2014

    I find that I still want to shoot myself while looking at log files.

Leave your comment

Guest Friday, 22 August 2014