High processing power.
GPUs are expensive because it takes an extreme amount of computing power to render realistic graphics that can produce lifelike visuals.
GPUs were initially designed for video game players, but they use more advanced methods than the classic circuitry found in computers and calculators. Specialized circuits had to be built with parts that could handle mathematical computations at blinding speeds. However, these types of parts are more expensive than those used in regular electronics. Computers also require a lot of fast memory (random access memory, or RAM), which is costly as well. All of this translates into higher costs for consumers.