What I Know, How I Know It, and What It Means to Me
This is my mantra as an expert witness. Read on to learn what I mean.
Requirements, Investigation, Analysis, And Opinion
Before rendering an opinion, a technology expert should first gather the information necessary to perform their analysis. The choice of methodology used for an investigation and analysis is dependent on many factors. When investigating engineering projects, it is often crucial to understand the context provided by the requirements imposed on the engineering team and the technology available during the timeframe of the project.
For the US federal legal system, Daubert is a flexible standard for the admissibility of expert witness testimony that permits novel approaches to analysis, as long as they are well-grounded in fact and present defensible reasoning. It follows that the methodology for distilling evidence from raw data must be defensible, and the results should be reproducible. As well, the methodology for analysis of the distilled evidence must be appropriate. When analyzing engineering projects, this means that the analysis used should be consistent with the project requirements, with due consideration for the technology available during the timeframe of the project, and the range of accepted practices at that time.
The US Federal Rule of Evidence 702 is similarly flexible.
The Expert Mantra
In light of Daubert and Rule 702, and from common sense, I suggest that technology expert opinions should be stated as follows, so they are defensible:
- What I know
- How I know it
- What it means to me
Let’s break this down.
What I Know
What I know is one or more observations gathered from a first-hand inspection of the evidence. For a technology expert faced with voluminous evidence, this often means running programs that extract and summarize information from the evidence. Useful evidence for a software expert might be source code, documentation, transcripts of conversations, or database contents.
I write programs as required, such as custom Microsoft Office macros, custom SQL commands, ad hoc command-line incantations, and/or custom shell scripts, etc.. I sometimes use commercial software tools such as decompilers and integrated development environments (IDEs).
Hearsay is not evidence, and an expert should take all verbal and non-authoritative written information under advisement.
How I Know It
Observations that cannot be replicated are suspect. This is why experts should always, in their reports, strive to document the exact sequence of commands that they performed to obtain the observations that they based their opinions on. This allows others to replicate the expert’s observations of the evidence, which means that their observations can be trusted.
How I know it provides credibility because anyone with access to the same evidence could type along with the expert report and verify that they get the same results.
Microsoft Office Skills
The legal profession relies heavily on Microsoft Office, in particular Word and Excel. While most attorneys are familiar with redlining the difference between two versions of a Word document, many are less familiar with style sheets, and few attorneys indeed are aware of Microsoft Office macros. To be effective when faced with a lot of evidence, experts should be able to develop custom Word and Excel macros. Some custom macros might be simple time savers for repetitive tasks, while other custom macros could be complex programs.
In a recent case, I wrote custom Excel macros to correlate two sources of information into a single timeline with approximately 20,000 entries, which was essential to understanding the history of a project. I could then opine that while the two document sources were known to be incomplete, the information found in each corroborated the other. I also developed a sense of how the individuals interacted with each other, and their technical and managerial strengths and weaknesses. It is amazing what a competent expert can discover when they know what to look for!
What It Means to Me
An expert’s opinion should be based on an analysis of their observations, or in other words what it means to me. They should strive to show how the observations were analyzed without getting lost in technical details. Even better, a confidence level in the results of their analysis should be expressed statistically. The degree of confidence is important because it informs their opinion.
Summary
It is essential for experts to be believable and easily understood. What I know, how I know it, and what it means to me is my mantra for providing value as an expert.