Use of Computer-Assisted Audit Tools and Techniques (CAATTs), Part 1

superfluitysmackoverSécurité

23 févr. 2014 (il y a 3 années et 4 mois)

66 vue(s)



http://www.theiia.org/itaudit/index.cfm?fuseaction=print&fid=380

Vol. 4, October 1, 2001

Audit Tools

Use of Computer
-
Assisted Audit Tools and Techniques
(CAATTs), Part 1

Readers' rating: 5 out of 5

By
Charles Le Grand
, CIA, CISA, CDP

CAATTs may be classified in the following groups:




Electronic Working Papers





Information Retrieval and Analysis





Fraud Detection





Network Security





Electronic Commerce and
Internet Security





Continuous Monitoring





Audit Reporting





Database of Audit History





Computer Based Training





Time Tracking

As audit tools grow more powerful and sophisticated, they are also becoming easier to learn and use.
And,
at the same time, they also must fit into a complex and ever changing environment. Features of
audit software can easily conflict with features of other software on the computer or network, and must be
carefully managed.

As tools become more powerful, audi
tors may use features or services provided in the software that
command considerable system resources (memory, processing cycles, communication bandwidth, and
storage) and compete with other users of those resources. For example, an auditor may request acc
ess
to a file with a program that will examine each record in the file and may lock other users out until the
process is complete. The processing could also require large amounts of network storage space at a time
when it is in short supply and could cause

a server to
crash
. It is important to schedule such processing at
times when other system users will not be delayed or prevented from performing their work. Alternatively,
many audit organizations perform their audit analyses using files copied or archive
d from the live
production files.

CAATTs may also be large, powerful, or specialized enough to require a dedicated server for audit
purposes. A server may be needed to support the audit website, or just to assure the independence and
security required by a
udit functions. And, as evidenced by the list of
software tools

attached to this
document, there are more tools available than the amount of time an auditor may have to learn

to use
those tools. So the need for software specialists to support internal auditing is increasing even as the
software is getting easier to use.

Risks associated with software tools and techniques

Software ease of use may also result in the implementati
on of features that unintentionally weaken
information security provisions. While software vendors may not be particularly open about their potential
weaknesses, a growing body of websites documents software weakness and available corrections. This
provide
s both positive and negative opportunities.

As weaknesses in software are discovered and documented, the vendors of those software products
develop corrections or
patches

that may be applied until the weakness is corrected in the next formal
release

versio
n of the software. However, many organizations do not apply such patches, for a variety of
reasons. Hackers know software frequently goes unpatched, so they search for particular versions of
software with known weaknesses. They may then launch an attack ag
ainst that system using software
developed to exploit known weakness. Such software, called a "script," may require little or no knowledge
to use. The successful attack using a script may give the attacker unlimited (or
root
) access to the target
system. N
ormally, root privileges are reserved for system administrators and are closely monitored. Once
an attacker has root access they have virtually unlimited access to the system, and may also obtain
access privileges to other systems with an established trust

relationship.

Another element contributing to risk in information systems and networks is the configuration of systems
as provided by vendors. Frequently systems are initially installed with the security and control features
turned off. System and network

engineers and administrators must select the appropriate mix of control
features they need and turn them on when the system is installed. Sometimes security and control
features will conflict with features of other system components or may add considerabl
e overhead to
system processing, such as through the use of system logging. When security components conflict with
operations, the typical response is to turn those components off. Unless the organization provides strong
security policy administration and/
or auditing, management may be unaware security features are not
being used. Therefore, frequent assessment and monitoring are important elements of information
security management.

The Center for Internet Security (CIS) (see
http://www.cisecurity.org
, a not
-
for
-
profit organization) has
developed benchmarks for identifying the security features that should be activated for specific operating
environments, and publishes the specific settings for individu
al operating systems. These
benchmarks

are available on their website. CIS also provides downloadable software to check system configurations
against the benchmark.

Electronic working papers

The capability to search for information in text, databases, or o
ther audit records is giving auditors great
ability to coordinate their efforts and to examine findings from prior or concurrent audits. The ability to
require standardized audit forms and formats can improve both the quality and consistency of audit
worki
ng papers. The management of current and archived working papers in a centralized audit file or
database can make it easier for audit management to coordinate concurrent audits and assure they
consider findings from prior or related projects.

Expert system
s provide an opportunity to add broad support and increased functionality to audit working
paper tools. For example, an expert system may evaluate responses to a questionnaire and automatically
generate links to additional related questions. Expert systems

may also look at patterns in information,
findings, recommendations or related concurrent or previous audits, and provide reports indicating
potential related or systemic problem areas.

As audit work paper tools provide the ability to include supporting i
nformation other than text or numbers


such as pictures, sound, and video


the methods for organizing and providing access to such
information must adapt accordingly. In future, auditors may discover that a great deal of information
needed in audit revie
ws may exist in forms other than text, numbers, or graphical characters.

A word of caution is in order: As you consider commercial solutions for managing electronic working
papers, consider the environment in which the software will operate. Some packages
require
environments that may be inconsistent with the systems and networks maintained by the rest of the
organization. Consider also flexibility. Some packages may be limited in the options available for different
types of working papers that can be used
and communicated among audit team members. Some
packages may need modifications to suit the needs of your organization. Modifications may result in
difficulty applying new releases of the software and/or may void the vendor’s warranty of features and
funct
ionality. These considerations are certainly not unique to audit software tools and are part of the
complexity routinely managed by information services professionals and management.

Information retrieval and analysis

To sample or not to sample

Historicall
y, auditors have relied on samples of transactions to perform their tests. With the use of
automated retrieval and analysis tools, it may be easier to assess all records than to evaluate a sample.
Furthermore, auditors can set parameters in software to ide
ntify all records meeting selection criteria. Full
selection of known error type records can eliminate the problem of
estimating

error rates. Instead, error
analysis can focus on those records with data that are outside the range of expected transaction va
lues
but still within the limitations that define error conditions.

Actual sampling techniques may be applied at the time records are selected from the production system,
or all records of a given type may be selected and sampling or more detailed selectio
n may be applied in
the analysis process.

Record selection criteria may be based on prior audits, but auditors should continuously assess
opportunities to improve audit coverage


especially if this can be accomplished at reduced overall cost.
Automated se
lection and analysis tools can facilitate improvements, but will not automatically assure such
improvements.

Retrieval and analysis software

Identifying and accessing information

Information retrieval and analysis tools can present significant technical ch
allenges to auditors as
information subject to audit may reside in diverse and distributed system types with varying degrees of
control and standardization. Data may be stored under the control of various machine types and
operating systems using differing

formats; it may move across telecommunications environments using
different protocols; it may be stored or archived by various database management systems using fixed or
variable length fields or records and subject to differing database standards; and it

may even reside in
numerous physical locations as in a distributed database or data warehousing environment. Particularly
sensitive data may only be available in encrypted form and may be subject to government regulations
regarding its transmission, stora
ge, controlling software, encryption key management, and import / export
or transmission across national borders.

Many auditing departments use technical specialists to locate and evaluate data sources and provide the
software tools to extract data and con
vert it into a form that can be used by audit analytical tools.
Because there are so many forms and formats for information and so many proprietary
standards

for
information storage, and because information systems environments change frequently, it may be

necessary to maintain significant technical expertise among the audit team members responsible for
using retrieval software. People with such expertise may be difficult to recruit or afford, and providing
training to audit staff for such skills may make t
hem highly marketable.

In some organizations or industries information is stored according to specified standards that do not
change frequently, and multiple audits may be performed on information in a common format. In such
cases libraries of information
retrieval routines can be maintained, accessed, and executed by any
auditor. In other organizations the frequency of change may be greater than the frequency of audits and
preparation of retrieval software routines may preclude the use of pre
-
programmed ro
utines.

Once information is stored in a form usable by audit analytical tools, auditors with varying degrees of
technical expertise may actually perform and review the results of analysis. Many ordinary office software
tools such as spreadsheets or databas
es may be able to access and analyze information stored in an
open database compliant (ODBC) format.

Some audit organizations not only maintain automated routines for information retrieval and analysis, but
they deploy such software via telecommunications
to allow reviews of remote systems without the time
and expense of staff travel. Organizations with centralized controls and standards management are best
suited to remote auditing, but auditors may also use some of the same types of software as deployed b
y
hackers to assess security and control in distributed systems environments without centralized controls.

Information analysis

Accumulation of information about business data over a period of time may allow analysis software to
identify patterns, shifts,
or trends in the data indicating changes in the business, the business
environment, the customer base, the economy, changing competition factors, etc. Such pattern analysis
may be important to business planning and competitive advantage, and may be perform
ed by groups
outside of internal auditing. However, if audit analysis recognizes such patterns then the auditors may be
able to provide a valuable contribution to the organization.

Audit analysis of data patterns may be focused on shifts that indicate a ne
ed to redefine record selection
criteria, quality management mechanisms, error threshold monitoring, or review of records and
transactions that fall outside the normal realm of events (possibly defined in standard deviations). But
audit analysis can also t
arget certain data patterns such as identification of artificial numbers. For
example, Benford’s Law defines a natural distribution of numbers common to all large bodies of numbers.
In circumstances where individuals make up or modify numbers due to fraud
or errors, the resulting set of
numbers will not follow Benford’s Law and may be detected and investigated via audit analysis software.
(For more information on this subject there are several articles in
ITAudit Forum’s

archives. Mark Nigrini
wrote a serie
s on Benford’s Law and Digital Analysis


published in the Emerging Issues department; and
Rich Lanza wrote an article on Continuous Monitoring


published in the Audit Tools department.)

More common audit data analysis routines include matching employee d
ata to customer or vendor
records, duplicate payments, payroll and overtime, approvals versus authorization levels, force codes,
system overrides, access authorities, telephone usage, and much, much more. Examples abound in
auditing literature.

Trends in i
nformation retrieval, analysis, and monitoring

A trend in auditor information retrieval and analysis is to include greater intelligence in auditing or
monitoring software embedded in business systems and networks. As auditors identify risk elements and
dev
elop software to detect errors, suspicious transactions, or unusual data patterns, it is often a relatively
simple process to embed such tests or monitors into production systems. In these cases, auditors can
then be informed of errors or changes in data p
atterns soon after they occur throughout the operating life
of the system or monitor.

Auditors planning to deploy embedded system audit features can be identified as "users" of systems
under development. Rather than functioning on the design and developmen
t team only as control
specialists, they function as any other system user or interfacing system representative. The auditors
specify the record selection and data format criteria for embedded monitors, as well as any special
features such as logging, or t
he ability to modify, expand, or suspend audit monitoring.

For example, auditors may expect certain systems to process transactions at expected volumes or within
certain monetary ranges. Embedded monitors may alert the auditor by triggering an alarm if tra
nsactions
exceed expected threshold boundaries and may gather and store copies of the related transactions. The
auditor can then evaluate the data and determine if the fluctuations are normal or require additional
appraisal. In either case, the audit softw
are may be provided additional logic or intelligence to enhance
such selection or appraisals in the future.

Typically, when audit monitors become more sophisticated than the tools used by managers responsible
for the systems, the managers will request that

they also be provided such functionality. After all, no one
wants the auditors to come in asking questions about problems before management is even aware of the
problems. As management controls and monitoring tools become more sophisticated to match or ex
ceed
the auditing tools, then auditors can shift their emphasis to areas of greater risk, or can increase the
sophistication or intelligence of their monitors. In either case, the overall control environment is enhanced.

In the future, the logic used by au
ditors to trace transactions and events forward and backward within
computer systems, networks, and files will also be embedded in sensitive systems. Then sensitive
transactions flowing through systems can carry with them embedded information indicating th
e source(s)
of the transactions and all routes taken through processing, networks, or files. Such "audit tags" will be
most useful in the case of monetary transactions such as payment processing or funds transfers and will
provide vital information needed
to detect or deter fraud.

With the decreasing costs and new capabilities of information processing and storage systems and
media, it is becoming feasible to capture and archive sensitive information at all points of entry,
processing, transfer, or storage.

The availability of "massive redundancy" in data management will enable
monitoring and analytical tools to track, in great detail, the changes applied to data throughout its life
cycle. Massive redundancy can also provide for data analysis using "voting"
and other analytical or
statistical methods. Thus appraisals of information integrity in the future could be based on complex data
analysis and proceed to controls analysis only as anomalies are encountered. This is the opposite of how
traditional audit ap
praisals are applied and may require some process reengineering within the auditing
profession.

About this article

This article is extracted from a paper prepared for an "International Seminar on IT in Audit" hosted by the
National Audit Office of China (C
NAO) September 16
-
21, 2001 in Beijing. The larger paper, titled
"Information Technology in Auditing," incorporates updated material from audit software articles originally
posted in the ITAudit Forum on September 1 and October 15, 1998. This article and th
e two subsequent
companion articles replace the older ones found in the ITAudit.org archives. An
updated list

of audit and
risk management software and related tools and serv
ices and their providers is also provided.

The IIA’s work with the CIAO and PCIS continues with the PCIS supporting the
"National Plan for Information Systems Protection" and working to facilitate information
sharing across sectors of the critical infrastr
uctures and extending outreach to other
nations to improve global security practices and help ensure protection of the global
economy.
For more information, or to participate in this activity, contact Charles
Le Grand at The IIA.



Vol. 4, October 15, 2
001

Audit Tools

Use of Computer
-
Assisted Audit Tools and Techniques
(CAATTs), Part 2

By
Charles Le Grand
, CIA, CISA, CDP

In
Part 1
, you saw that CAATTs can be classified into 10 groups. There, you reviewed the first two:
electronic working papers, and information retrieval and analysis. Here in Part 2, you’ll study the
remaining eight classification
s: fraud detection, network security and performance, electronic commerce
and Internet security, continuous monitoring, audit reporting, database of audit history, computer
-
based
training, and time tracking.


Fraud detection

Areas most frequently identifie
d by auditors for fraud detection include accounts payable, employee
payroll, expense reporting, and inventory management. Historically, auditors have looked for typical fraud
indicators such as duplicate payments for invoices or expense reports, invalid v
endors, unusually high
payments or payments exceeding authority levels, payroll payments to former employees, or detectable
patterns in inventory "shrinkage." In recent years software has provided auditors with tools that can also
identify unexpected or un
explained patterns in data that may indicate fraud.

Benford’s Law, as indicated in
Part 1
, provides several rules that apply to large bodies of numbers. One
example is the per
centage of numbers in a population that will begin with the numerals "1" or "2." In
normal populations, fully 30% of the numbers will start with the numeral "1" and the percentage of
occurrence will decrease rapidly as the numerals increase from "2" throug
h "9." By using analytical
software, such as ACL that can apply Benford’s Law to a body of numbers, the auditor may be able to
detect fraudulent or "artificial" numbers because people making up or manipulating numbers typically do
not know about Benford’s
Law. A simple example of record types that may be detected through such
analysis is purchase orders generated for amounts just below an individual’s authorization limit. Rather
than generating a purchase order for $25,000, which would require a higher leve
l of approval, a person
may generate several purchase orders just under their $5,000 authorization limit. Data analysis would
detect an anomaly for an unusually large group of purchase order amounts beginning with "4."

Network security and performance

Netw
ork security software is typically used by network administrators. However, as the incidence of cyber
attacks perpetrated via networks has increased dramatically in the past few years, auditors have found
the need to add network security assessment softwar
e to their tool kits. Sometimes auditors will use the
same tools as those used by their network administrators. Other times auditors will use their own
specialized tools. Auditors may even use common hacker tools. In any case, it is important for auditors
to
coordinate closely with network managers and administrators because tests and scans can adversely
impact network performance if used improperly. Improper use of network security analysis tools can also
cause a network to fail, ceasing operation until th
e network administrator is able to resolve the problem
and restore operation. Causing a network to crash could result in considerable costs as well as lost
productivity, revenues, and opportunity for an organization.

Among the most important features of an
y network are availability and performance. So auditors must
exercise extreme caution in ensuring their assessments and analysis do not impact network performance
or availability. (See
Audits From Hell

in
ITAudit Forum

.) There are also examples where an organization’s
shareholder value was negatively impacted because the organization was the victim of a destructive
cyber attack or even a simple distributed denial of service (D
DOS) attack.

In some cases network security assessment and analysis software may be provided free or at a low cost
by organizations hoping to sell security services or other products. However, auditors must be wary of
free software, particularly the variet
y known as
freeware
. It may be difficult or impossible to understand
the full functionality of such software, the impacts it may have on systems and networks, and the integrity
of its processing. Furthermore, without a legitimate vendor, there is no recour
se for problems that may be
caused by the software, and user support may be difficult or impossible to obtain.

Network assessment and analysis software may be used to map the full extent of a network. Sometimes
a device on a network may be modified to act
as a bridge or gateway to other networks. In such cases
network administration and management may not be aware of the full scope of the network and may
apply inappropriate security provisions.
Network traffic analysis software
, sometimes called
sniffers
or

supersniffers
, may be used to analyze and/or capture messages or even individual keystrokes in
network traffic. Such tools, if improperly used, can violate security, confidentiality, and privacy rules, but
they can also be used to monitor and enforce info
rmation security policies and legal or regulatory
requirements.

More important than the tools used by auditors for network assessments and analyses are the tools that
make up the entire security environment for the organization and its networks. Network se
curity tools
include firewalls, intrusion detection systems, worm and virus protection, backup and recovery, traffic and
pattern analysis, encryption, public key infrastructure (PKI) and certificate authority (CA) administration,
access control and monitor
ing, vulnerability assessments, and much more. It is pointless to focus on
individual components in a network security environment without addressing the full control system. For
example, firewalls can provide good controls but are ineffective if they do n
ot properly apply security
policies or if their coverage is incomplete. And, virtually all security provisions can be circumvented by
social engineering if employees are not adequately instructed and monitored in applying good security
practices.

Electroni
c commerce and Internet security

Electronic commerce via the Internet has increased at an explosive pace in recent years. Most
organizations have implemented
business
-
to
-
business (B2B)

and
business
-
to
-
consumer (B2C)

e
-
commerce systems using Internet tools.

Competition and opportunity are driving forces for this growth.
But rapid growth in an area of new technological developments inevitably introduces new problems and
escalates the significance of some older problems.

The Internet facilitates communications

via e
-
mail. Today, e
-
mail is the standard for the rate of progress
and responsiveness for virtually every organization. Similarly, browsers and websites set the standard for
providing information about an organization and its products and services. And, i
n many cases, the
website is the vehicle for delivery of information, products, and services.

To be useful, information must be available, but this availability puts it at risk. Connectivity makes
information available when and where it is needed and is th
e nature of doing business today. Because
organizations are linked through the Internet and other public networks to suppliers, customers, and
business partners, they are also connected to virtually everyone else in the world. Connectivity exposes
informat
ion to risks outside the organization’s control.

In the modern world, everything that business or government does with its information technology
becomes part of the global information infrastructure. Organizations must build infrastructures to a very
high

standard. Attaching weak components to the infrastructure puts your organization


as well as your
neighbor’s


at risk. Responsible citizens will contribute only sound components to that cooperative
infrastructure. Therein lies the essence of the auditor
s’ involvement in providing assurance of the security
of information and systems operating in connection with the Internet.

E
-
commerce tools for auditors are just beginning to emerge. Generally, auditors are using the same tools
as systems administrators,
information security professionals, and even hackers. An organization
concerned about its security may employ auditors or others to assess system security using
tiger team

tactics


authorized attempts to break into their systems. In many, if not most, cas
es such attacks are
successful and provide management with information about various ways outsiders can break into
systems or insiders can exploit system security weaknesses. Non
-
invasive tools are also used to probe
networks for security flaws that might
be exploited. New tools are also being introduced that will evaluate
the configuration of security features in key network components such as the operating system, firewalls,
intrusion detection systems, virus protection systems, and more.

E
-
commerce tools

also include encryption, public key infrastructures (PKI) and the related certification
authorities (CA) that facilitate the distribution and validation of encryption keys and related services. A key
feature of being able to conduct business over the Inte
rnet while being assured of a valid agreement and
protecting privacy is obtaining the services of third
-
party trusted agents. Assessment of PKI, CA, and
third
-
party trust features built into systems, networks, and business operations is beyond the capabili
ties
of most auditors today. Notable exceptions


auditors who must be fully capable of addressing e
-
commerce systems, security, controls, and assurance auditing


include those auditors working with
organizations who are the leaders in implementing Intern
et e
-
commerce systems. Such organizations
include major banks and related financial institutions, credit card providers and processing entities, large
manufacturing organizations engaged in B2B and/or B2C commerce, leading technology providers, and
similar
ly advanced organizations.

However, as previously noted, advancements in e
-
commerce are occurring at an accelerating pace. E
-
business is becoming synonymous with business. The automated tools and techniques being developed
and deployed by the leaders today

will become standard assurance and auditing techniques used by
auditors at all levels in the near future. A factor contributing to the increased capability of auditors in e
-
commerce will be the demands by boards of directors, insurers, and regulatory bodi
es for improved
assurance of effective and continuous information security.

Continuous monitoring

Continuous monitoring in systems and networks will be a byproduct of the increasing demand for
immediate and continuous access to reliable information by mana
gement, owners, investors, and
regulators of organizations of all types and sizes. The pervasive availability of electronic communications
drives the demand for reliable information and related assurance services.

Integrated accounting systems are rapidly
becoming commonplace, and will soon be the established
basis for the expectation of timeliness in availability of financial information. Immediate financial reporting
and availability of information for comparison and analysis are becoming byproducts of in
tegrated
applications across all areas of businesses and industries


combining operational and financial
information in integrated databases and management reporting. The emergence of standards such as
extensible markup language (XML) and the related exte
nsible business reporting language (XBRL) will
also help to accelerate the pace of increasing expectations for the availability of information and the
related assurance of its integrity.

As previously indicated, advancements in information monitoring and a
nalysis are being accelerated both
by increasing demands for timely and accurate information, and by advances in technology that
contribute to the intelligence, capabilities, and timeliness of monitoring and analysis systems. Continuous
monitoring systems
are not new, but they also cannot be considered widespread at this time.
Nonetheless, the advances in systems and the increasing expectations of information availability will
ensure that continuous monitoring and auditing systems will be the rule rather th
an the exception in the
near future.

Audit reporting

Some audit tools today provide automatic linking between work performed, information gathered, auditor
assessments, and information used in or supporting audit reports. Intelligent work papers may note
a
nswers in internal control questionnaires (ICQ) that indicate actual or potential weaknesses and
automatically prepare a section in the audit report to document the weakness and/or resolution of the
problem.

Audit reporting, too, can automatically provide
information about sections of audits performed by
individual auditors as they are completed so the audit supervisor will know the ongoing status of audit
projects. Such reporting will also allow the supervisor to concentrate on audit processes that indicat
e
problems and/or provide additional resources in areas falling behind schedule.

The audit report can easily contain links to working papers, worksheets, graphs or other information that
will be automatically updated as data changes. Report files can be sh
ared by audit team members and
management by implementing simple controls over access such as read
-
only access to those not
authorized to change the files.

Audit reports can be distributed in electronic format via e
-
mail, file transfer, or audit website. I
n such
cases, auditors must assure appropriate security, confidentiality, and access controls for such reports.
Encryption technology is rapidly developing and will become the standard mechanism for electronic
message integrity, sender and receiver authent
ication, and access control.

Database of audit history

The audit history database should provide a historical perspective for all audits on the plan or schedule.
Audit history can identify recurring or unresolved issues or problems, or indicate areas of ri
sk.
Furthermore, many sections of audit work papers can be copied from prior files and updated to save
auditor time and effort.

Audit reports can be indexed by key words to facilitate review or searching, or may be searched in their
entirety depending on t
he techniques employed. Similarities in data patterns, audit findings, or
recommendations can be found using indexing or search technology, and can support expansion or
reduction of audit scope.

The technical delivery of the audit history database may be b
ased in database management system
technology or may be delivered via a website. Regardless, it is also important to consider confidentiality
of audit information and provide access controls and other privacy and security techniques for files and
communica
tions. Audit assessments of controls can represent a risk element because they could provide
information needed to identify control weaknesses.

Computer
-
based training

Embedded training and help features are included in most audit software tools today. Man
y software
providers and other organizations offer both generic and specific training for the use of software tools.
However, computer
-
based training (CBT) can span the broad realm of auditing, as well as activities
subject to auditing, and should not be l
imited by previous experience. Training can be informal and self
motivated, or it can be a formal element of audit administration providing feedback to the trainee as well
as to audit management.

In the context of CBT as an audit tool, it is most likely to

be self motivated. It may be limited by the time
and tools available, the speed at which the tools operate, or the auditor’s energy, imagination, and
exposure to information. For example, if auditors do not have access to the World Wide Web, then they
can
not use it to search for information. If their access path is slow and/or expensive, then the time
requirements may quickly outpace the value received or reduce the auditors’ enthusiasm for such
learning. If traveling auditors do not have remote access to
their central files or e
-
mail, then they cannot
search audit histories and cannot use a list server to seek input from others on a problem or question.

Ultimately, audit management, and of course the budget, will determine the tool set provided to auditors
,
but the auditors themselves will determine how effectively the tools are used. Training should focus on
how to seek out and learn new information and approaches, not just on how to perform previously defined
tasks or use existing software features.

Time
tracking

In some cases, it may be possible to direct internal system clocks to record the time auditors spend using
their computers and track that time to individual projects. It may also be relevant to record the time and
resources used by programs as the
y process for the purposes of individual audit projects. Eventually,
automated tracking of resources will become the norm, but today it is more likely to provide input only to
the time tracking and management processes.

An audit management system can provi
de detailed and summarized analyses of productivity and other
reporting parameters required to effectively manage an auditing department. Time tracking and reporting
can be elements of the project management system previously described, and can be used to
evaluate
performance, estimate time requirements for scheduling, and relate critical skills to their most effective
deployment.

About this article

This article is extracted from a paper prepared for an "International Seminar on IT in Audit" hosted by the
N
ational Audit Office of China (CNAO) September 16
-
21, 2001 in Beijing. The larger paper, titled
"Information Technology in Auditing," incorporates updated material from audit software articles originally
posted in the ITAudit Forum on September 1 and Octob
er 15, 1998. This article and the two subsequent
companion articles replace the older ones found in the ITAudit.org archives. An
updated list

of audit and
risk management sof
tware and related tools and services and their providers is also provided.

The IIA’s work with the CIAO and PCIS continues with the PCIS supporting the "National Plan for
Information Systems Protection" and working to facilitate information sharing across
sectors of the critical
infrastructures and extending outreach to other nations to improve global security practices and help
ensure protection of the global economy.
For more information, or to participate in this activity,
contact Charles Le Grand at The

IIA.


Guidelines for Requesting Data from Computer Systems

The following guidelines will save time and improve the chances for successfully obtaining and
testing computer data.

I. PLAN FOR THE REQUEST

Before requesting computer
-
generated data from IS dep
artments, you should have the following:

1. A basic understanding of the computer system, including the purpose of the system, who uses
the system, what data elements (or fields) are available, what reports are routinely generated, and
what the data is use
d for.

2. An audit plan for reviewing or testing the data, including why you are testing the data, who
will test it, and what other files will be required.

3. The name and phone number of: 1) the person responsible for maintaining the system; and 2)
the pe
rson responsible for creating the computer data in response to your request.

To help understand the data in a computer system and identif~r exactly what data elements
(fields) you will need for testing, you must obtain and review the appropriate DATA
DICTI
ONARY or file layout. The dictionary should provide information such as the name,
source, purpose, and a narrative explanation of each data element in the system.

II. REQUEST THE DATA IN WRITING

Once you have the above information, you are ready to make yo
ur data request. The request
letter, usually signed by a manager or above, should include the name of the data elements
requested as they are identified in the data dictionary. Request only those data elements that are
relevant to your audit test; never re
quest a copy of all the data elements in the system, unless
they are all needed to complete your planned test.

Your request letter should include:

~ The date by which you need to have the data;

~ The name and phone number of a person to contact if there ar
e any questions regarding the
request;

~ A list of data parameters, such as specific transaction codes or a cut
-
off date for the data;

~ The format in which you want the data; for example, .dbf, .wkl, flat ASCII or EBCDIC files;

> The media on which the da
ta is to be put; such as, disk, tape, download, etc.

~ The name and the phone number of the auditor requesting the data; and

~ The name and address to which the data should be sent.

It is very important that the client provide, in writing, the total number

of records in the database
and the dollar amount (control totals) for all
-
important numeric fields.

Attachment I provides a list of technical specifications and documentation requirements that the
client should use when providing computer data to you. You

should provide a copy of the
checklist to the client and request that they complete the list and forward it to you with the
computer data. Failure to include these specifications may cause a delay in processing the data.

III. AVOID POTENTIAL PROBLEMS

To r
educe the probability of delays in processing your data, you should be aware of the
following general rules.

1. 1. Be cautious with print files. Print files are usually a copy of data listed on hard copy reports
that is stored as a computer file. According
ly, they often contain data such as headers, footers
and subtotals that are shown on reports. If you do request a print file, you should also request
some pages of the hard copy report. Also remember that the data in the report file has already
been proces
sed. Your test of the original data could be compromised if you limit yourself to just
report files.

2. Request fixed length files. Fixed length means that each record in the file has the same number
of characters. If the client cannot provide fixed length

files, you may have to perform additional
steps to import the data into IDEA.

3. Verify that the client provided the required documentation. Incomplete documentation is often
the cause of problems in processing computer data. Accordingly, we recommend tha
t you verify
that the client has provided all the needed documentation and that the data is in the format you
requested. If it is not, you should immediately contact the person responsible for providing the
data.

4. Microcomputer files can usually be impor
ted into IDEA. However, there are a wide variety of
possible formats. Some formats can be troublesome. For this reason, if the client plans on giving
you data in microcomputer format, .dbf files (dbase format) are the easiest file formats to import
into ID
EA.


ATTACHMENT I

TECHNICAL SPECIFICATIONS FOR COMPUTER DATA

1. Storage Medium:

3480, 3490, or 3490E Cartridge

9
-
track, 6,250 bytes per inch

Floppy diskette

CD
-
ROM

Network Server

Other (please explain):

Is the file compressed? Yes No

2. Data Specifications
:

File Format:

EBCDIC

ASCII

Dbase

Other (please explain):__________________________________________

File Type:

Fixed Length File

Variable Length File

Field Separator______________________________

Record Delimiter(s)_____________________________________

Str
ing Encapsulator__________________________

3. Required Documentation:

a. Record layout that includes:

The beginning and ending position of each data element in the system;

Each data element's width; and

Each data element's type, such as character, numeric
with sign embedded, or alphanumeric, etc.

b. Name and phone number of person(s) responsible for creating and providing the file.

c. File Name (Data Set Name)

d. Total number of records in file.

e. Control totals for important numeric fields.

Vol. 3, July

15, 2000

Standards

Auditing Online Computer Systems

By
John Yu
, CDP, FCGA

A
s previously reported, in March 2000 the International Audit Practice Committee (IAPC) of
IFAC

released an exposure draft on four topics which form a supplement to ISA (International Standard on
Auditing) 401 “Auditing in a Computer Information Systems Environment (CIS).” The four topics are:




CIS Environme
nts


Stand
-
Alone Microcomputers





CIS Environments


On
-
Line Computer Systems





CIS Environments


Database Systems





Computer Assisted Audit Techniques


In a
previous article
, I reviewed the exposu
re draft on standalone microcomputers. In this article, I’ll
review the exposure draft on On
-
Line Computer Systems.

Online computer systems

The exposure draft defines online computer systems as computer systems “
that enable users to access
data and program
s directly through terminal devices…
” This definition is sufficiently broad as to cover all
forms of online systems, including the traditional smart server/dumb terminal variety, as well as the
client/server variety because the definition covers all possib
ilities.

Contrary to the impression many people have, traditional dumb terminals still run a significant number of
the world’s CIS environment. These range from terminals used by travel agents and older generation of
point of sale (POS) terminals for many
retail businesses, to terminals used in airline check
-
in counters
and those used to run most of the legacy systems used in many corporations. The exposure draft
describes two classes of terminals:




general purpose terminals

such as basic keyboard/screen, i
ntelligent terminals that can
perform a certain amount of data validation, and microcomputers





special purpose terminals

such as POS devices, automated teller machines, and voice
response systems such as those used in telebanking


While these two classes
cover a number of terminals used in online systems, they fail to recognize many
more modern (and advanced) terminals. The following are some examples of devices used in online
systems not covered by the definitions in the exposure draft:




biometric devices

used for authentication (for a more detailed description of biometrics,
see “
Application of Biometrics
”)





network computers such as Sun’s JavaStation





Internet devices or e
-
appliances, such as pe
rsonal digital assistants (PDAs), WebTV, i
-
opener, various net
-
phones, and net
-
cars (for a more detailed description of e
-
appliances, see “
What auditors should know about e
-
appliances
”)


All
these devices operate in an online environment as “terminals.”

Types of online systems

The exposure draft suggests five types of online systems:




online/real time





online batch





online memo update





online inquiry





online download/upload


Online/real t
ime systems are the classic online systems where transactions update the master file
immediately.

Online batch systems are those with online data capture but batch updates.

Online memo update is defined as “
On
-
line input with memo update processing, also k
nown as shadow
update, combines on
-
line/real time processing with on
-
line batch processing. Individual transactions
immediately update a memo file containing information that has been extracted from the most recent
version of the master file. Inquiries are

made from this memo file. These same transactions are added to
a transaction file for subsequent validation and updating of a master file on a batch mode.
” According to
this description, the transactions only update a copy of the master file, without affe
cting the actual master
file. The master file is affected only when the transactions are posted later. For all intents and purposes,
this form of online system is really a batch system.

Online inquiry systems restrict the user to perform queries only.

By t
he description in the exposure draft, online download/upload sounds like another variation of the
online memo update system where the memo file is a copy of the master file downloaded to the terminal.
After it is updated locally, it is then uploaded back t
o the original master file for updating.

The section on “Characteristics of On
-
Line Computer Systems” (paragraphs 18 to 22) seems to be a
hodge
-
podge of comments without any particular focus.

Internal control issues

As can be expected, this exposure draft
devotes significant time to internal control issues. In fact, two
topics (“Internal Control in an On
-
Line Computer System” and “Effect of On
-
Line Computer Systems on
the Accounting System and Related Internal Controls”) are devoted to these issues. While t
he coverage
of internal control issues is reasonably comprehensive, the placement of certain paragraphs seems odd
at times. For example, under the second topic, I found a passing reference to risks of viruses. The issue
of risks associated with viruses sho
uld be given more prominent coverage under the general discussion
of internal controls rather than specifically on accounting system controls. Coverage of firewalls and
hacking should also be strengthened.

Effect of online systems on audit procedures

The e
xposure draft makes the point that it is “
more effective for the auditor to perform a pre
-
implementation review of new on
-
line accounting applications than to review the applications after the
installation.
” Here, the focus is on “on
-
line accounting applic
ations,” and seems rather narrow.
Increasingly, e
-
commerce businesses are relying heavily on online sales systems that are focused on the
sales and marketing side of the business, and yet such sales and marketing applications are more
important to the busi
ness than the accounting applications, which the auditors ignore to their own peril. In
any case, often, auditors need to audit online systems after they are implemented, playing no part in the
implementation.

Some reference should be made to auditing onl
ine transactions that involve third parties. This is
particularly the case with some e
-
commerce sites where the online credit card processing is handled by
an agent or service provider authorized by the bank external to the e
-
commerce site.

Overall, the e
xposure draft makes a good attempt to bring the standard up
-
to
-
date. The only major flaw is
that it has not gone far enough to deal with an increasingly complex online e
-
commerce environment that
provides auditors with new and special challenges.

The IAPC
will accept comments and suggestions up to July 31, 2000.

Computer Assisted Audit Techniques

Readers' rating: 4 out of 5

By
John Yu
, CDP, FCGA

A
s I previously reported, in Ma
rch 2000, the International Audit Practice Committee
(IAPC) of IFAC released an exposure draft on four topics which form a supplement to
ISA (International Standard on Auditing) 401 "Auditing in a Computer Information
Systems Environment (CIS)." The four t
opics are:



CIS Environments


Stand
-
Alone Microcomputers




CIS Environments


On
-
Line Computer Systems




CIS Environments


Database Systems




Computer Assisted Audit Techniques

Author’s note: Although this set of exposure drafts was published in March with c
omments due by July
31, 2000, a final version of these practice statements has not yet appeared on the
IFAC Web site

as of
early November 2000.

To review the first three articles on the exposure draft, see "
Auditing Standalone Microcomputers",
"Auditing Online Computer Systems", and "Auditing Database Systems." In this article, you’ll learn about
the last topic, CAATs.

According to the exposure draft, the purpose of the statement on CAATs "…
is to provide guid
ance in the
use of Computer Assisted Audit Techniques (CAATs), which are techniques that use the computer as an
audit tool."

The exposure draft "
applies to all uses of CAATs involving a computer of any type or size."

As with the other three topics, this se
gment of the exposure draft reads like a tutorial on CAATs, devoting
a substantial amount of space describing the basics.

Description of CAATs

Paragraph 5 provides examples of where CAATs may be applied when performing various auditing
procedures. These in
clude the traditional data analysis procedures, as well as the use of any computer
means in any aspect of an audit. To illustrate, one of the examples cited is the "
creation of electronic
working papers by downloading the general ledger for audit testing
."

The "
use of expert systems in the
design of audit programs and in audit planning and risk assessment
" is also considered a form of CAAT.
However, in light of the importance of e
-
commerce in this day and age, at least one e
-
commerce example
should have bee
n included in the list.

Paragraph 6 lists various CAAT tools, but these two paragraphs (this one and the preceding one) are
poorly organized. The list in Paragraph 6 consists of various types of computer programs that can be
used in CAATs (package programs
, purpose
-
written programs, utility programs, and systems
management programs). The rest of the list consists of descriptions of various test data techniques. This
disjointed presentation is confusing. It is better to organize the material on test data tec
hniques into its
own paragraph.

Paragraph 7 describes "
evolving techniques that emanate from using the power and sophistication of
microcomputers, particularly laptop computers…,
" then goes onto provide examples that do not
specifically apply to microcompu
ters and laptop computers. One of the techniques attributed to the power
and sophistication of microcomputers is "
expert systems, which can design specific tests for use by the
auditor.
" You might well question the validity of this statement. In any case,
the narrow distinction made
between "microcomputers" and "laptop computers" in this paragraph is an obsolete view of the computing
world. In the client
-
server model and the Application Service Provider (ASP) model, there is no need to
make the distinction
between the workstation and the server, both forming an integral computing unit to
the user.

Manual tests

Paragraph 12 focuses on the impracticality of manual tests where there is lack of hard copy evidence.
This paragraph takes a negative approach and des
cribes conditions under which manual tests cannot be
carried out, implying that there is no other choice but to use CAATs. This reflects old school thinking, in
which examining hard copy audit evidence is still considered the primary auditing method. Incre
asingly,
as organizations embrace the Internet as a means of conducting their business externally and internally,
there will be no hard copies. CAATs should be used by all auditors as a standard approach to auditing.

Using CAATs

Paragraphs 18 to 26 describ
e various steps required to use CAATs in a mainframe environment despite
earlier statements in the exposure draft describing CAATs as the use of any computing means in carrying
out an audit. Therefore, this narrow focus on mainframe environments where CAAT

programs are run
against the auditee’s data files is inadequate when providing a full and accurate description of how
CAATs should be used.

Several references are made to the need for the cooperation of the auditee’s IT staff, stating the obvious.
But the

exposure draft provides no guidance on how to proceed if cooperation is not forthcoming.

Paragraph 21 states that the "
presence of the auditor is not necessarily required at the computer facility
during the running of a CAAT to ensure appropriate control
procedures
." This statement is puzzling. If the
auditor relies on the auditee’s staff to run CAAT procedures, what is there to prevent manipulation or
distortion of the results?

Using CAATs in small business computer environments

Paragraph 27 deals with th
e use of CAATs in a small computer environment. This paragraph, as it
currently stands, provides little guidance on what constitutes a "small computer environment." Another
example of incomplete guidance is "
in cases where smaller volumes of data are proce
ssed, manual
methods may be more cost
-
effective."

There is no direction on what constitutes "smaller volumes of data"
such that manual methods may be better.

Furthermore, the points raised in this paragraph again reveal antiquated thinking. To illustrate,
one of the
points raised states "
certain audit package programs may not operate on small computers, thus restricting
the auditor’s choice of CAATs."

There are a number of powerful CAAT tools that can work with virtually
any type of data files from computer
s of any size. ACL is an example of such a tool.

Using CAATs in e
-
commerce environments

The exposure draft is silent on this very important area. More guidance should be provided. Some of the
audit techniques developed in the AICPA WebTrust program could b
e incorporated.

Dated approach

Of the four topics in the IAPC exposure draft on the supplement to ISA (International Standard on
Auditing) 401 "Auditing in a Computer Information Systems Environment (CIS)," the material on CAATs is
the most dated and requi
res a more innovative approach.