Cohen, J. A coefficient of agreement for nominal scales. Educational and psychological measure 1960,20, 37-46. This technical report provides detailed information on the reasons for using a common computer computing program (Microsoft Excel®) to calculate different forms of interobserver agreement for continuous and discontinuous datasets. In addition, we offer a brief tutorial for using an Excel table to automatically calculate the traditional total number, partial match in intervals, exact tuning, trial test, interval interval, multiple interval, total duration and average duration of interobserver duration algorithms. We conclude with a discussion of how practitioners can integrate this tool into their clinical work. Holley, J. A. and Guilford, J. P. Note on the agreement`s G index. Educational and psychological measure 1964,24, 749-753. An approach to improve the accuracy of the agreement between two observers for interval recording is simply to limit agreement analyses to cases where at least one observer has recorded a target response in the meantime.
Intervals in which no observer has reported a target response are excluded from the calculation in order to provide stricter agree statistics. Cooper et al. (2007) suggest that the IOA point interval (also known as “deposit agreement” in the research literature) is most advantageous when targeted responses are at low rates. In the figure 2 examples, the second, third and fourth intervals are ignored for calculation purposes, as none of the observed intervals have been answered at these intervals. As a result, IOA statistics are only calculated from the first, fifth, sixth and seventh intervals. Since only more than half of the intervals (5th and 6th intervals) have been stopped, the approval rate is 50% (2/4). Hawkins, R. P., and Dotson, V. A. Reliability scores that delude: An Alice in Wonderland trip through the misleading characteristics of interobserver agreement scores in interval recording. In E. Ramp and G.
Semb (Eds.), behavioral analysis: areas of research and application. N.J.: Prentice Hall, 1975, 539-376. Harris, F.C. and Lahey, B.B. A method to combine event and non-attendance assessments. Journal of Applied Behavior Analysis 1978,11, 523-527. House, A. E., Farber, J. and Nier, L.
L. Accuracy and speed of reliability calculation using different ratios of the Interobserver agreement. Lecture in Postersession, Association for Advancement of Behavior Therapy, New York, November 1980. Intervall-for-interval IOA. In short, the interval interval method assesses the proportion of intervals in which both observers agreed to determine whether the target reaction occurred. Note that this implies agreement on attendance and lack of response. This is calculated by adding the total number of agreed intervals to the sum of agreed intervals and divided at regular intervals.