In our long quest to reduce false alarms in IP security camera detections, we've tried and tested a large range of detection methods over the years, from physical PIRs to the most advanced AI-based intelligent video analytics (IVA) on the market today.
Early 2021 Axis Object Analytics (AOA) became available in some Axis mainstream camera models. Now, after about a year of using, testing, and deploying AOA-based detection, we can say with confidence that AOA works, and works well for the purposes of minimising false alarms. We have seen such consistent performance that we are confident in specifying AOA-capable cameras in most scenarios where alarms are to be raised reliably and persons notified.
The main camera model we have used for AOA thus far is the Axis P1455-LE outdoor bullet IP camera.
Detection Analytics and AOA
Implementing 3rd-party IVA in a large or complex software-based video management system (VMS) such as Milestone XProtect or Axis Camera Station can be complicated, which can lead to troublesome issues and long periods spent configuring, calibrating, validating, maintaining and supporting these complex integrated systems, and so we were cautiously optimistic when Axis announced some time ago that with their ARTPEC-7 chipset, they would be releasing an advanced analytic which takes advantage of the chip's additional processing power to offer an improved human and vehicle classification and detection algorithm. As said, this new facility is called Axis Object Analytics or 'AOA'.
There are caveats, of course. No solution is 100% free of false alarms and, most importantly, the conditions for the accurate working of AOA must be met for it to work well. However, we have found that even poorly implemented setups of AOA can have a relatively high accuracy.
AOA comes as a free plugin (known as an ACAP) on each supported Axis camera and this can be used alongside other ACAPs and even detection methods, so long as the camera has enough CPU power. It runs on the camera itself (at the 'edge') and signals the VMS when a detection is made. Currently, AOA can detect humans and vehicles, but the list of objects detected will increase.
There are two types of AOA analytics, machine learning (with MLPU) and deep learning (with DLPU) and soon, with the release of Axis' ARTPEC-8 chipset there will be only deep learning variants, which theoretically should improve performance and usability even further. AOA classifies the moving objects it sees and notes their movement in relation to 'zones' created by you and when those zones are breached by either classified object type, the alarm is raised. The idea being that if an unclassified object such as a fox or a cat crosses the zone then the alarm is not raised, whereas with many other detection methods and algorithms, this movement may cause a false alarm.
What is AOA with Machine Learning?
AOA does not use AI. Let's get this out of the way. It does not learn on-the-fly, adapt to changing conditions or improve its classifications over time (except with firmware upgrades). However, the machine learning versions of AOA have been trained against many images (some thousands) of objects they should classify and then use an ‘average’ of these detections to classify new objects it sees. We have focused our deployments on the more affordable machine learning AOA variants until now.
AOA with machine learning will classify only humans and vehicles generically.
What is AOA with Deep Learning?
Again, deep learning is not AI. Deep learning-based AOA has been trained with a far larger number of pre-classified objects, sometimes millions, hence being 'deep' and offering even greater classification accuracy. The Deep Learning Processing Unit built into these cameras also contains some clever acceleration technologies which allow accurate classifications to be made quickly. Deep learning AOA is rapidly coming down in cost and will become the dominant camera type that we, NW, deploy and sell.
AOA with deep learning will classify humans and various vehicle types such as trucks, buses, cars, motorbikes, and cycles.
What triggering conditions can be used?
The AOA application allows for two main types – zone entering or line crossing. With zone entering you map out a ‘zone’ of coverage or an area where you want to be notified if someone enters. With line crossing you simply draw a line and set a detection direction.
Zone-entering scenario screenshot
Line crossing scenario screenshot
How does AOA perform against other industry-leading IVA products?
In our real-world deployments, we have noted a similar level of performance between Axis cameras with high-end, third-party IVA installed and those with Axis’ freely included AOA in use. Third-party IVAs can also run as an ACAP on Axis cameras but may also require direct software integration into the VMS, which AOA does not, reducing the maintenance burden during the system lifecycle. On this, we would make the point that we do not recommend using AOA where another IVA solution may be specified as part of a tender, as AOA is not i-LIDS approved, and may not fully meet the requirements of the tender specification.
In our use cases where AOA setup conditions were the same as where we would implement an expensive, third-party IVA, the accuracy of AOA was as good, if not better than we would expect from an i-LIDS approved IVA. This includes a minimal number of false alarms. We have found that third-party IVAs (or third-party ACAP) can be difficult and time-consuming to set up for the lowest false alarm rate particularly where insects and water droplets are concerned, and so we have been impressed at how well AOA has performed with very limited calibration requirements.
It's important to also observe that third-party IVAs (or ACAPs) may have additional capabilities that AOA does not have, so a direct comparison is not truly possible. However, when used for sterile-zone protection from humans and vehicles AOA performs well and may often be a better option to implement due to cost and complexity factors.
How to set up Axis Object Analytics?
AOA has some specific conditions to be met for best performance. These are noted both in the camera's help files and in the Axis Object Analytics online manual and the most important ones are discussed below. As far as the setup process goes once the camera is installed Axis have made the setup process simple, but there are a few gotchas.
The basic setup of AOA involves simply turning the ACAP on and selecting whether to classify for humans, vehicles, or both. However, hidden away in the settings tab is the perspective calibration tool which, we strongly recommend, you use in every case. This tool allows you to set the perspective of the scene and the height of known objects within it and set the perspective using three height bars on the scene. Doing this will reduce your false alarms (particularly from small animals) and increase your detection and classification accuracy enormously.
Screenshot of the perspective calibration tool
Metadata
If you are using a VMS such as Milestone XProtect you can also decide whether to use the metadata channel from the camera (which contains the classification and detection information) in the VMS. On top of this, you can decide whether to 'burn in' the zone and detection information into the video stream which your VMS records. This can be useful for operators to confirm detections. With Milestone XProtect you can perform filtered searches based on this metadata and only show human, or vehicle detection in search results.
Screenshot of video burn-in
This screenshot shows a great deal of camera movement (yellow boxes) but still accurately detects (red box) the human entering the zone (red bounding box).
What are the pre-conditions for installation of a camera using AOA?
The AOA manual (linked above) goes into detail on mounting considerations, but our findings go into more detail. However, the very basics are shown in the image below:
- Mounting height
- Tilt
- Detection area
- Minimum detection distance
- Maximum detection distance
- Camera mounting height affects detection distance and accuracy. The best mounting height seems to be around 3-3.2M, any higher and accuracy may drop as the camera looks down from a greater angle, which makes it more difficult for the application to classify objects. Traditionally, many installers mount cameras higher than this, so considerations need to be made for the safety of the camera itself
- Illumination is key – a well-lit scene with high contrast is necessary for accurate detections and minimal false alarms. In some scenes, the best performance will need a b/w image (IR-cut filter turned off)
- Tilt angle - the camera must be sufficiently oriented towards the ground so that the centre of the image is below the horizon. Mount the camera so that the minimum detection distance is longer than half of the camera’s mounting height (minimum detection distance > camera mounting height / 2)
- Camera must be level – zero roll angle
- Sterile zone – the fewer obstacles in-scene there are, the more accurate AOA will be
- Camera stability – camera movement has a big impact on accuracy. For example, if a camera is mounted to a lamp post, which we strongly advise against, but many people do, the camera will move in high winds and see stationary cars as moving, so will trigger detections on that (and these are not false alarms as far as the camera is concerned! It would be a false alarm because of wrong installation)
- There is a detection blind spot beneath the camera - accurate classification does not happen when the subject is very close to the camera
- No occlusions - needs the full human figure or vehicle, not half a body
- Accurate detection takes a little longer when it is raining (rain affects the object definition but ultimately accuracy is preserved)
- Avoid strong fore lighting - position the camera away from strong lighting or headlights shining into the lens
- Use exclude areas where possible to avoid some false alarms
Key considerations
The key points we have found in our real-world usage are below. Follow these guidelines and you can expect a high accuracy of object classification and detection with a low number of false alarms.
How does AOA perform in inclement weather?
In February 2022 we went through storms Dudley, Eunice and Franklin in the space of a week or so. Eunice had particularly high winds and rain and even during this weather event, AOA performance was unaffected.
What other benefits are there beyond accurate detection for alarm-raising?
Intelligent video analytics isn’t just about making detections of unwanted people and vehicles, though that is their primary function. IVA can help with efficiency and cost-saving as well.
For example, VMS systems can and do have expensive amounts of storage. A well-set-up IVA can significantly reduce the amount of storage required on a system by only allowing recording to take place, at least at higher frame rates, when a detection is made. At all other times, there can be either no recording at all or recording at a lower frame rate, thus saving on storage costs and expensive disk drives and server chassis.
Bandwidth is also a consideration and AOA can be implemented to save on bandwidth. For example, a VMS can be set to pull video from a camera only when it makes a detection. In the case of Milestone XProtect, this is achieved by using the metadata channel to signal Milestone to record when a detection is made.
As AOA is an edge-based IVA – i.e. it works on board the camera itself – this can help to reduce the overall complexity of a system with complex requirements. Other IVA solutions introduce additional software components or need additional physical server resources. These other IVAs may well be more suitable for your needs, so I stress again that AOA is no substitute for all IVAs and the use of a particular IVA is governed by each individual use case.
Reliable IVAs allow CCTV operators to focus their attention elsewhere and attend to other tasks than watching key cameras around the clock. Most VMS can allow for cameras to pop up when a detection is made, grabbing the attention of the operator who may be doing other things at the time.
AOA can ‘burn in’ both the zone and its various object detections into either the video that is recorded or the video that is shown live to the operator. This can help the operator to determine if something needs to be investigated more quickly.
Bear in mind as well in your requirements whether you need analytics in a real-time scenario (perhaps for generating alarms) or in a non-real-time scenario (for example for recording and retrospective viewing only). The use of any analytics should be put through this filter to help in determining whether the analytics fits the needs of the organisation or surveillance scenario. For example, when deploying video analytics in a non-real-time way where it is paramount that all activities are captured, choose your analytics carefully. False negatives (missed detections) can arise from intelligent video analytics which is not set up correctly or have not been carefully matched to the detection scenario. In cases like this consider using an additional, 2nd detection method such as VMD or even continuous recording, to ensure that the events you want are reliably captured. You can easily set this up with
Milestone XProtect. In addition, in bad weather, intelligent video analytics should be combined with other detection methods to ensure a secure system.
Overall, we’ve been impressed at how well AOA has performed, but mindful of the fact that this can only be achieved with strong planning, adherence to the conditions Axis set for its deployment and attention paid to its setup and calibration.
We look forward now to seeing more cameras with ARTPEC-8 and Deep Learning becoming mainstream during the course of 2022, expecting AOA performance to improve further still.