HowTo Parse log records of ACI APIC
For those of you following me on Twitter, this is how it started:
Every time I have to search for something inside the Event menu of the Cisco APIC (System > Events), I’m complaining about the lack of search options to find something useful across these thousand pages.
Here is how it looks from the GUI, notice the lack of search option:
In order to be able to parse the events without spending hours scrolling over these GUI pages, I had to connect in SSH, extract the multiple eventRecord.xml, eventRecord1.xml… (follow this TAC guide to see how to do it: Collecting Basic TAC outputs such as Audits, Faults , Events), extract the files via WinSCP and parse them manually. The problem is that the .xml files are not correctly formatted (when you open the file, it’s just a single line file) so I had to format them manually first, before being able to parse them, and without mentioning the fact that formatting and parsing took a few seconds for each file, which was obviously not efficient.
From a batch console, I was using xmllint to format, and grep to search for specific words, specific time of events.. Here is an example:
> cat eventRecord-0.xml | xmllint --format - > formatted-eventRecord-0.xml
> cat formatted-eventRecord-0.xml | grep -i "2021-01-21T12:00"
xmllint is a part of libxml2-utils on debian, this is what I personally use with WSL (Windows Subsystem for Linux) see http://www.xmlsoft.org/ (also available for windows natively).
But this solution is not efficient enough, I had to find a better way to parse these files. And after starting my studies for DEVASC a few days ago (that’s a story for another post), I tried to develop a small, useful script to help me parse these .xml files.
Cisco ACI Log Parser
Let me introduce you to Cisco ACI Log Parser, Available on my Github account.
The script is developed in Python 3, using the os, sys and lxml modules, two functions, and a few lines of code.
It takes a word as an argument you want to search inside all the .xml files in the same folder of the python script. This argument can be a specific word like “error” or a part of a timestamp like “2021-01-20T” to search all the logs that occurred on the 20th of January.
After taking the argument, the script starts checking all the .xml files available in the same folder as the python script and prettyPrint them with lxml module in order to change the single line files into well-structured XML files.
This is the function I found to do it:
def prettyPrintXml(xmlFilePathToPrettyPrint):
"""Save/Replace the .xml file after prettyPrinting it"""
assert xmlFilePathToPrettyPrint is not None
parser = etree.XMLParser(resolve_entities=False, strip_cdata=False)
document = etree.parse(xmlFilePathToPrettyPrint, parser)
return document.write(xmlFilePathToPrettyPrint, pretty_print=True, encoding='utf-8')
Then for each file, the script is checking each row for the word passed in arguments, and prints the complete row if the word appeared.
Here is the function that searches for the argument inside each line, and if a match is found, returns the entire line:
def lines_that_contain(fp, string):
"""Return lines that contain a specific string"""
return [line for line in fp if string in line]
Here is how it looks in action, notice all the .xml files listed with the parser.py in the same folder:
> ls
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 22/01/2021 00:07 14072521 aaaModLR.xml
-a---- 22/01/2021 00:07 59552484 eventRecord-0.xml
-a---- 22/01/2021 00:07 61179794 eventRecord-1.xml
-a---- 22/01/2021 00:07 67249806 eventRecord-2.xml
-a---- 22/01/2021 00:07 68979198 eventRecord-3.xml
-a---- 22/01/2021 00:07 152 eventRecord-4.xml
-a---- 22/01/2021 00:07 61178 faultInfo.xml
-a---- 22/01/2021 00:07 92325611 faultRecord-0.xml
-a---- 22/01/2021 00:07 152 faultRecord-1.xml
-a---- 22/01/2021 00:07 167 faultRecord-2.xml
-a---- 22/01/2021 00:07 167 faultRecord-3.xml
-a---- 22/01/2021 00:07 167 faultRecord-4.xml
-a---- 10/01/2021 23:37 1666 parser.py
The script is used by adding a word or series of words, and you can send the output into a file if you want as I did here, if you don’t, the output will be printed directly.
I was surprised by how fast it is to parse a few hundreds of megabits with this method compared to the previous one (and I’m sure the code can be wayyy more optimized…).
> python parser.py "2021-01-20T15:44" > MyParsedFile.txt
> cat MyParsedFile.txt
+--------------+
| aaaModLR.xml |
+--------------+
Lines inside aaaModLR.xml matching the word: 2021-01-20T15:44
+-------------------+
| eventRecord-0.xml |
+-------------------+
Lines inside eventRecord-0.xml matching the word: 2021-01-20T15:44
<eventRecord affected="topology/pod-1/node-109/sys/tunnel-[tunnel35]" cause="oper-state-change" changeSet="ope....
<eventRecord affected="topology/pod-1/node-103/sys/tunnel-[tunnel10]" cause="oper-state-change" changeSet="ope...
[...]
+-------------------+
| eventRecord-1.xml |
+-------------------+
Lines inside eventRecord-1.xml matching the word: 2021-01-20T15:44
+-------------------+
| eventRecord-2.xml |
+-------------------+
Lines inside eventRecord-2.xml matching the word: 2021-01-20T15:44
+-------------------+
| eventRecord-3.xml |
+-------------------+
Lines inside eventRecord-3.xml matching the word: 2021-01-20T15:44
+-------------------+
| eventRecord-4.xml |
+-------------------+
Lines inside eventRecord-4.xml matching the word: 2021-01-20T15:44
+---------------+
| faultInfo.xml |
+---------------+
Lines inside faultInfo.xml matching the word: 2021-01-20T15:44
+-------------------+
| faultRecord-0.xml |
+-------------------+
Lines inside faultRecord-0.xml matching the word: 2021-01-20T15:44
+-------------------+
| faultRecord-1.xml |
+-------------------+
Lines inside faultRecord-1.xml matching the word: 2021-01-20T15:44
+-------------------+
| faultRecord-2.xml |
+-------------------+
Lines inside faultRecord-2.xml matching the word: 2021-01-20T15:44
+-------------------+
| faultRecord-3.xml |
+-------------------+
Lines inside faultRecord-3.xml matching the word: 2021-01-20T15:44
+-------------------+
| faultRecord-4.xml |
+-------------------+
Lines inside faultRecord-4.xml matching the word: 2021-01-20T15:44
Feel free to use it if you like, and let me know if it helped you on your day to day troubleshooting of ACI. Don’t hesitate to help me improve it, I’ll try to make it faster and add new features as I should continue to discover a few things along the DevNet Associate studies :)
Benoit
Network engineer at CNS Communications. CCIE #47705, focused on R&S, Data Center, SD-WAN & Automation.
Follow Me:
2 Comments
Comments are Disabled
Why not just use https:///api/node/class/aaaModLR.json?order-by=aaaModLR.created|desc&page=0&page-size=1000
page-size can be adjusted
Thanks for your comment Brian, sure, the page size can be adjusted.
I only copied the recommended commands from the Cisco TAC Article here.