PDA Logo.gif (6595 bytes)

When Getting the Audit Done Is the Only Thing


our services

about Peter Davis+Assoc.


security/audit info

Privacy Test

Security & Audit Tools


Windows NT Server IIS

Windows 95


Java, JavaScript and ActiveX

Intrusion Detection Systems

Security Industry Shakeout

Securing Groupware

Client/Server Audit: One Bite At A Time

Configuring Cisco Denial of Service Security Features - Part 1

Configuring Cisco Denial of Service Security Features - Part 2

Configuring Cisco Lock-and-Key

Configuring Cisco Reflexive Access Lists

Dysfunctional Controls: Useless, Impractical, Inefficient and Poorly-Designed

TCPA: Who Can You Trust?

When Getting the Audit Done Is the Only Thing

Palladium: Friend or Foe?

Commentary: Quis Custodiet Ipsos Custodes?

Data Management: Data Destruction and Preservation

Security & Audit Products
Top Ten Security Links 
Security & Audit Checklists
Computer & Security
Security & Audit Bibliography 
Search Page

legal info

privacy info

Dateline: Toronto, ON, August 2002

Last winter, I delivered a one day security session at a conference in London, England. My topic was “Securing Your Place on the Web” and I had about 10 attendees. In a competing session in the room next door, there were at least fifty participants. That session’s topic was “Hacking Web Applications.” Mind you the speaker for the other session was excellent, and may have put a few extra bodies in the seats, but his ability to deliver an entertaining session didn’t account for the difference. At the time, I remarked to several colleagues that this phenomenon was indicative of a malaise sweeping the audit and security professions. People don’t want to learn about the underlying technology, they just want to know what to do to complete the audit and wow their bosses who also might not understand the technology.

I find further evidence when I do audit courses. As is my practice, I always ask class participants what they hope to get out of the 2-, 3- or 5-day course. Invariably, someone, either the first or second person, asks for the audit program. I usually tell them the whole course is the audit program. Oops, there go the evaluations!

I find further evidence in some of the more technical security and audit courses. These courses always have a prerequisite of networking in general or TCP/IP in particular. But as a rule, I find that the majority of the class does not have the requisite background. Who wants to know the OSI model when I can run Whisker? Why learn about half- and full-duplex when you can learn about hacking applications. The latter is infinitely more entertaining!

I feel I am seeing, in these brief exchanges, the combined success and failure of our efforts over the years. We’re helping people complete audits; we’re answering people’s questions, instead of questioning their implied assumptions. We’re also providing automated tools with built-in expertise so our professionals don’t have to acquire the expertise themselves. Until now, I have thought that this was a completely good thing, but I’m starting to have my doubts.

The problem, I’m starting to suspect, is that auditors may have learned to resist the idea of absorbing a foundation of information before they start accumulating details. We’ve thrown so much complexity at people, during the past few years, that auditors have had to develop a defence mechanism: “Just tell me what I need to know!” But when we do this, we wind up with people who are merely following checklists and audit programs that might as well be written in Latin. A case in point from my days with a public accounting firm may help to illustrate. One audit cycle, I picked up the working papers from the previous year’s audit. Previously, the auditor had written down that the client had RACF, a mainframe access control product from IBM. When I logged on to the system, I quickly determined they did not have RACF installed. I found the system programmer and explained what I had found. The system programmer looked me straight in the eye and said “We have RACF” as he pointed to the tapes and manuals sitting in his cubicle. No one stopped to ask them whether they had installed RACF; they just asked them whether they had RACF or not.

The problem doesn’t just lie with the auditors themselves. In the past, staff had a chance to learn the fundamentals, and maybe even see opportunities to do things in fundamentally different ways, before they were forced to buy in to the existing way of doing things, before they felt in danger of being hopelessly overtaken by minutiae. And supposedly helpful tools such as the World Wide Web compound this problem by providing lots of information, misinformation and disinformation. It’s like drinking water from a fire hose.

Look, for example, at the way we’ve changed our approach to the task of auditing. When I started working for a bank, my superior asked me to research SMF and to make recommendations regarding the records we needed to keep and the correct parameters for the start-up files. I spent a few days researching the matter and learnt quite a bit about SMF (simplistically, a MVS log for those you don’t know) that I remember to this day. Nowadays, given the same assignment, someone most likely would look for an automated tool that reviewed the SMF parameters and made recommendations. The auditor doesn’t learn anything about SMF, has no idea how it works, and doesn’t figure out the questions to ask the system programmer.

The problem also strikes in the opposite direction. Sometimes, it’s not a question of knowing too little, but rather of “knowing” too much. Should you only answer the question that a person chooses to ask, you give up any opportunity to influence the assumptions and beliefs behind that question. For example, should someone ask me how to find out what services are running on their Windows desktop, I could answer: “Use the netstat command at the command prompt” and that person could happily go back to work, now that I had “solved” the “problem.” But perhaps, instead of responding directly to the question, I should have asked: “What are you trying to find out?” But would that person appreciate my question? Not likely. People tend to get peeved when they think someone is condescending by asking, “Are you sure that’s really what you want?” But you don’t do anyone a favor when your tightly focused answer helps the questioner keep doing the same irrelevant things; and instead of getting them out of their rut, lets them continue making “progress” in the wrong direction.

Even a brilliant methodology like CobiT contributes to the problem. With CobiT methodology, you have domains, sub-domains, control objectives and audit guidelines. Heck, you even get a list of the people to interview and the documents to review. Everything is laid right out in a straightforward fashion for you to follow. The structure leads to standardized audits with standardized working papers. You can have someone with minimal subject matter expertise use the expertise of CobiT to do the audit. However, this also means the auditors are less likely to do any lateral thinking. Most likely, they won’t do any task or ask any question not in the methodology itself. They just want to rush to the end of the domains, so they can finish.

It’s ironic in the information age, that the automated tools are actually helping us stay stupid and uninformed; merely because we can now find the answer that we don’t know better than to want. The checklist and audit program mentality stops people from thinking outside the box. Perhaps, if we had people thinking outside the box, we might have prevented Enron, WorldCom and Adelphia.

Abridged version of an article published in EDPACS by Auerbach Publications 2002.

Tell a friend about this page!
Their Name:
Their Email:
Your Name:
Your Email: