Unlocking black boxes in legal disclosures

In summary

• Traditional technology tools and in-house expertise are often insufficient to extract and analyse the complex data some cases rely on
• Data analytics experts can make sense of these “black boxes”, ensuring vital data is protected, analysed efficiently and used effectively
• These experts can collaborate closely with the client to ensures better results, combining business insights with analytical expertise to identify clues more quickly and effectively

In legal disputes, disclosure is not just a procedural obligation; it’s a cornerstone of fairness. It ensures that all parties have access to the relevant facts, promoting transparency and upholding the integrity of the judicial process.

But it also brings challenges in managing the large volumes of information involved: thousands of emails and hundreds of documents in even the simplest of cases. For years, technology has helped with this task, preventing information overload.

However, the disclosure of large, structured datasets from finance and operations systems can present significant challenges to the principles of transparency and fairness. The opposing party may provide hard drives full of XML, DBF or BKP files, or other unexpected formats –  gigabytes of data with little context. Technology tools that work well for emails and documents can struggle with these large and unhelpful structured files. 

Dealing with these black boxes of complex data can be particularly challenging, especially when they include older or proprietary formats. This is where external expertise is essential to manage the risks of using and relying on the data incorrectly or being unable to use it at all.

Even modern open formats like XML, which are generally easier to handle, come with their own challenges. Familiar tools like Excel can be problematic, requiring repairs to spreadsheets or testing of ambiguous formats. Reliance on an inexperienced repairer (or one who fails to appreciate the context of the data) risks destroying evidence and perhaps the entire case through incorrect assumptions or failure to consider possible explanations objectively.

Expertise and experience are vital in identifying issues with the disclosure, and these can impact whether the data can be relied upon. Sometimes gaps are intentional, but as ever, we should never put down to malice what can be explained by ignorance or mistake. Nor should it be assumed the system the data came from is infallible. Overwriting data is a real risk, and orphaned records or partially cancelled transactions are not uncommon, to give just two examples.

Expert data forensics specialists will account for all the issues and provide robust checks to detect and evidence any incomplete disclosure or manipulation after the fact.

As with any data analytics project, following a structured approach in these cases is essential. The Medallion model is one such approach. It involves organising data into layers (bronze, silver, and gold) to stratify data quality and integrity checking. This includes mapping data lineage back to the source, documenting any transformations made, and reviewing any later findings against transformation risks, which is essential when later relying on outputs or findings in litigation.

A partnership is also critical to success. Both the client's business expertise and effective data analytics are key to identifying patterns of behaviour. Extending the relationship beyond the extraction of the data to an investigative collaboration will avoid errors and delays in the analysis. An independent view is also beneficial, whether to bring professional scepticism or more openness to benign explanations. Being too transactional and fully outsourcing the process can miss easy wins and opportunities for practical solutions that emerge in collaboration.

Crucially, the outputs need to be appropriate and practical. The goal is to flag issues and find clues early that can help the case, rather than building a perfect, comprehensive system that arrives too late to be useful. Working together as a team to make sense of the data and quickly identify the key areas can make a big difference. With the right experts and a collaborative working relationship, opening a black box can be key to unlocking the case.