In this chapter, based on results of test theory, rough set theory, and exact learning, we study arbitrary infinite binary information systems each of which consists of an infinite set of elements and an infinite set of two-valued functions (attributes) defined on the set of elements. We consider the notion of a problem over information system, which is described by a finite number of attributes: for a given element, we should recognize values of these attributes. As algorithms for problem solving, we study decision trees of three types based on attributes from problem description: (1) using only attributes, (2) using only hypotheses, and (3) using both attributes and hypotheses. As time complexity, we use the depth of decision trees. In the worst case, with the growth of the number of attributes in the problem description, the minimum depth of decision trees of type 1 grows either as a logarithm or linearly, and the minimum depth of decision trees of types 2 and 3 either is bounded from above by a constant or grows as a logarithm, or linearly. Based on these results, we divide the set of all infinite binary information systems into four complexity classes.