ISO/IEC DIS 18520

Description / Abstract: This document identifies the reference framework for the benchmarking of vision-based spatial registration and tracking (vSRT) methods for mixed and augmented reality (MAR).

The framework provides typical benchmarking processes, benchmark indicators, and trial set elements that are necessary to successfully identify, define, design, select, and apply benchmarking of vSRT methods for MAR. It also provides definitions for terms on benchmarking of vSRT methods for MAR.

In addition, this document provides a conformance check sheet as a tool to clarify how each benchmarking activity conform to this document in a compact form by declaring which benchmarking processes and benchmark indicators are included, and what kinds of trial sets are used in each benchmarking activity.

The target audience of this document are stakeholders of benchmarking activities. The followings are examples of how this document can be used directly or indirectly:

— by a benchmarking service provider, a benchmark provider, or a benchmarking competition organization who wants to align their benchmarking activities including self-benchmarking and open/closed competitions to be consistent with this document;

— by a technology developer/supplier who wants to estimate and evaluate the performance of a vSRT method for MAR appropriately with a benchmarking service provider, a benchmark provider, or a benchmarking competition organization who aligns their benchmarking activities to be consistent with this document in order; or

— by a technology user who wants to obtain the benchmarking results based on a benchmarking activity, which is consistent with this document, to compare the existing vSRT methods for MAR in terms of the performance.