Think Big. Start Small. Preparing for Machine Learning.

Imagine how powerful it will be if we can upload and analyze images of subjects committing crimes to automatically fill out a report for us?

Would it be faster and more accurate in the long run?

Where we’ve started

When our customers report crime, they fill in the usual stuff like height, build, and gender. There’s a text field asking for more information on appearance. Text fields allow people to write as little or as much as they have time. It is entirely dependant on how confident the reporter is at remembering incidents and translating them into well-written descriptions. 

Is it accurate? Is it fast? And more importantly, is it consistent?

Ask people to describe an object. Is it big or small? What color is it? Do you describe the same object using the same words all the time? Through the data, we can see that people describe the same things in many different ways. This is perfectly normal, but there is huge potential for improvements.

Don’t take my word for it, let’s take a look at the data.

Two immediate benefits of structured data: simplicity and speed

• Looking at the graph above, a collective 76.25% of people have one of the four selectable attributes. Compare this to the unstructured fields (person description and methodology) where they have a collective 25.5% usage.
• This definitively proves people are more likely to use a readily available checkbox rather than type out a narrative.
• The speed of getting useful information from the data is also better when it has structure. Making the most of unstructured data requires expertise and specialized tools e.g. natural language processing, machine learning etc. These subjects are intrinsically complicated, time consuming and therefore expensive (cost and effort).
• Conversely, having a little structure can speed up search results immensely. E.g. with our fuzzy search implementation we can retrieve all people with red beanies quickly.

Putting our user experience hat on

Writing up crime reports is time consuming. Writing them well even more so. We compared reports from experienced Auror users against newer users. The quality of reporting is quite different, with experienced Reporters consistently writing better reports than newer users.

Writing is a core skill for security personnel roles. What could we do to make this process better for non-security personnel?

Structured data

An early hypothesis was that structured data could be captured faster than writing and would help construct better quality narratives. Couple this with a great user interface we felt that we will capture better quality intel. We also felt that if there were a range of options presented to reporters then it could trigger a memory, for example ‘Beard, Shoes, Bags’ may trigger a memory that could be critical.

A huge benefit of structured data is that it’s easily indexable, which makes it incredibly searchable! This means more accurate matches along with a huge gain in speed.

Structuring the content

Through data analysis we could clearly see how we could break down appearance to 7 main categories:

1. Head
2. Upper body
3. Lower body
4. Feet
5. Accessories
6. Bags
7. Scars or tattoos

We set up a spreadsheet of all the possible attributes that could match each category and we ended up with something like this:

Early designs

User experience is the phrase we use to describe how something performs for our customers. It’s a way of saying - we made this for you, not for us.

The early designs were well received by our users. The use of icons and labels on the buttons invited them to add content.

Assumptions tested

• We had the right amount of content to collect.
• Color is the most important secondary information to include.
• Breaking down the different parts of the body is the most efficient way to add information.
• The user interface is easy to use.

User experience testing

We conducted some in depth testing with a variety of users and customers. 

Here’s what we learned:

• Scars and tattoos are the most important distinguishing features to capture. It was recommended that we split these out into two options.
• There was a lot confusion when entering in different types of clothing and accessories. We noticed that users would spend a long time looking through each category in order to find a match. It was recommended that we put clothing into a single category.
• There are more important attributes than color that needed to be captured. It was recommended that we change this to a short free text field. This would allow reporters to add descriptions of logos or branding.
• There wasn’t a proper way describe facial features. It was recommended that we include these features.
• There was heavy reliance on using the ‘Other’ category. With the challenges outlined above, some reporters were using the fallback category to input items. While we loved that they persisted it clearly demonstrated that we had some work to do.

Updating the designs

We reduced the amount of categories to 5 which had a remarkable effect on reducing the cognitive load. The new appearance function suddenly became more powerful whilst being quick and easy to use. Nice!

‍

So what does this have to do with machine learning?

Our customers want to report crime fast, and accurate data to solve and prevent crime. We need to be able to teach the machines what is good data and what isn’t. Remember when Facebook used to tag everyone’s faces to ask you if this is Person X or Person Y? That’s machine learning at scale. The user interface is the convergence of the two, so let’s make this the best, most simple process possible.

We recognize that when all the amazing capabilities of machine learning are unlocked, it will take some time for people to adapt, trust, and believe that the outcomes are better than people themselves.

We’re preparing to humanize the interface between machine learning to help report, solve, and prevent crime.

How we imagine it could work...

Our new reporting forms start with uploading photos first. We see this as the start point for asking the following:

Is this John Doe? – 97% match based on facial features and beanie. Confirm (Yes/No)

Head –Dark beard. Confirm (Yes/No)

Clothing – Black beanie with with NYC logo. Confirm (Yes/No)

Clothing – White jacket. Confirm (Yes/No)

Height – 182cm / 5’ 11”. Confirm (Yes/No)

Build – Muscular. Confirm (Yes/No)

We expect incident reporting to become faster, more accurate, and most importantly, a magical experience for our customers. We can’t wait.