Patents in Virtual/Augmented Reality: Discussions in Software Patents

Hae Chan Park Edited by Gwen Ha

Hae Chan is from the Bay Area. He is a freshman at UC Santa Barbara studying Electrical Engineering. He plans on pursuing law school after achieving his undergraduate degree, eventually culminating his career in his goal of becoming a patent/IP lawyer.

ABSTRACT

The metaverse, a general term used to encompass various virtual reality platforms, is a relatively new technological concept in the realm of patents. Regardless, numerous companies believe in the untapped potential of cyberspace as a business, resulting in patentees finding interest in the medium. This article will cover the aspects of U.S. law that influence the patents within virtual reality by reviewing sections of the United States Code that explore patents. After establishing current codes regarding patent eligibility, major events, and landmark cases regarding patents in virtual worlds will be examined. Contemporary examples of virtual/augmented reality-related patents will then be discussed. Current precedents and their implications for the metaverse will be considered. Historically, patents have generally focused on patent-eligibility through hardware means, even with software patents. However, recent history in patents involving virtual reality has circumvented this trend, being nearly entirely virtual and software-based, implying that future metaverse patents will not require unique hardware aspects.

12

INTRODUCTION

Virtual reality, as well as similar branches such as augmented reality, virtual worlds, and conceptualized metaverses as pitched by newly branded company Meta, has been a rapidly growing market for technology-based industries. Data suggests that virtual reality (VR) and augmented reality (AR) investments are projected to multiply 21-fold by 2022.1 Arguments discussing the legitimacy of VR as a profitable market continue, however, most of the developed world has begun to have a rudimentary understanding of the novel field. Most modern technology companies have embraced the new medium, making them no exception in the acceptance of VR and AR as viable markets. In 2022, billionaire and Meta CEO of Meta, previously Facebook, declared that their upcoming product promptly named ‘Metaverse,’ will become the ‘holy grail’ of interactive media experiences.2 The release of VR and AR has also introduced controversial legal issues regarding patents along with its success. Prominently, intellectual property law, a segment of the broader category of patent law, discusses the majority of the relevance of the topic.This is due to VR and AR’s reliance on computer programming and software, which is the specialization of intellectual property regulations. This can pose problems if VR and AR platforms are used commercially by non-developer users such as players or othercompanies as developers have protections on the product, complicating issues even further. On top of copyright ownerships, many software companies are notorious for not filing patents in the industry, with only

1 Iberdrola Corporativa, Virtual Reality: another world within sight. https://www.iberdrola.com/innovation/virtualreality#:~:text=THE%20FUTURE%20OF%20VIRTUAL%20REALITY,15.5%20billio n%20euros%20by%202022, (last accessed April 16, 2022). 2 Kara Carlson, At SXSW, Mark Zuckerberg says metaverse is 'Holy Grail' of social experience, Statesman (March 16, 2022, 9:30 AM), https://www.statesman.com/story/business/2022/03/16/sxsw-facebooks-markzuckerberg-says-metaverse-future-internet/7051230001/

13

landmark cases such as Apple’s spatial audiofor VR being prevalent.3 However, the greatest issue with VR-related patents is the complications that arise due to the fact that much of the patents regarding VR are softwarebased. Claimed software patents are difficult to deal with as the regulations covering hardware patents do not necessarily apply to software patents, making it difficult to categorize these claimed patents into actual patents or not.

As a result, this article will discuss this problem, as the issue will only expand due to the ever-growing nature of VR and AR as well as the software field as a whole. Data on current patents and landmark cases will be analyzed to answer what traits VR and AR software and hardware constitute as patentable. Section I of the article will cover the important aspects of U.S. Patent Law seen within 35 U.S.C (United States Code), such as regulations in software patents and their application to VR and AR. Section II will discuss major landmark cases and other important court cases in the field to establish a foundation of the USPTO’s views on patents. Section III will finally analyze current software and hardware patents and how the USPTO has evolved to adapt to VR and AR.

I. PATENT LAW

The overarching goal of a patent is for an individual or organization to disclose information on an invention in exchange for ownership of the invention over a predetermined time period, which is often 20 years. A patent is defined as an exclusive right to an invention according to the World

3 Jack Pucher, Apple wins a Patent relating to Spatial Audio in context with Augmented Reality and a Future HMD, Patently Apple (August 24, 2021, 5:37 AM), https://www.patentlyapple.com/patently-apple/2021/08/apple-wins-a-patent-relating-tospatial-audio-in-context-with-augmented-reality-and-a-future-hmd.html

14

Intellectual Property Organization.4 Additionally, patents remain relevant to this day as they are often mutually beneficial, providing others with knowledge of a unique invention and also providing the creator protection on commercial use. The regulations regarding patents are covered in 35 U.S.C. Within the United States Code, sections 100 through 103 pertain the most heavily to this topic.

A. SECTION 100

Section 100 establishes the primary definitions relevant to patents that will be important for the following three sections. The following terms are defined in this section: “invention,” “process,” “United States,” “patentee,” “third-party requester,” “inventor,” “joint inventor/co-inventor,” “joint research agreement,” “effective filing date,” and “claimed invention,” wherein the United States describes the territories and states where these regulations are effective. An invention is defined simply as an “invention or discovery,” which is straightforward. A process is defined by the U.S.C. as a “process, art or method, and includes a new use of a known process, machine, manufacture, composition of matter, or material.” This describes the technical information regarding the patented invention, which is important to applications. A patentee is defined as “whom the patent was issued,” as well as “the successors in the title to the patentee.” The definition clarifies that patentees are not necessarily always the individual who invents the invention, but simply the individual who owns the patent. A “third-party requester” is defined “as a person requesting ex parte reexamination under section 302 who is not the patent owner.” An individual who intends to challenge an extant patent filed by the patent office would be defined as a third-party requester. An inventor, another straightforward term, is defined as “the individual or, if a joint invention, the individuals collectively invented

4 Patents, World Intellectual Property Organization, https://www.wipo.int/patents/en/, (last accessed April 8, 2022).

15

or discovered the subject matter of the invention.” Through the definition of ‘inventor’, the definition of ‘converter’ is practically defined as well, who is just anindividual who “invented or discovered the subject matter of a joint invention.” A joint research agreement is “a written contract, grant, or cooperative agreement entered into by 2 or more persons or entities for the performance of experimental, developmental or research work in the field of the claimed invention.” A claimed invention is simply a “subject matter defined by a claim in a patent or an application for a patent.” Regarding the final definition that was not covered, the effective filing date does not pertain to the topic at hand.5 However, the rest of the terms will prove to be important to understand for the remainder of this article.

B. SECTION 101

Section 101 is the first major section of the 35 U.S.C, specifically saying, “Whoever invents ordiscovers any new and useful process, the machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.” This section essentially describes the types of inventions that are eligible or ineligible for a patent.6 There are four major categories in this section: process, machine, manufacture, or composition of matter. As defined previously, a process is described as a series of methods or steps.A machine is defined as an object made up of several components such as a computer, which is composed of multiple microcontrollers and integrated circuits. A manufactureris an object that is created from raw materials and a composition of matter is essentially just a product that is made up of two or more substances, which can be a liquid/chemical mixture. On top of these categories that the invention must fall under, the invention must also be a “new and useful improvement”

5 Definitions, 35 U.S.C § 100 (2011). 6 Inventions Patentable, 35 U.S.C § 101 (2011).

16

technology, which does not justfall under the field of engineering. Eligibility for patenting must fulfill three conditions: the claimed patent must be substantial, the claimed patent must be substantial in research (no further research needed), and the claimed invention cannot have a trivial utility attached to it in addition to its credibility.

C. SECTION 102 AND 103

Section 102 discusses whether or not the patent in question is a unique invention. Specifically, it is stated that an invention must be novel when compared to previous prior art searches.7 This categorizes one of the primary conditions for patentability. Similarly, section 103 discusses another condition for patentability, which is based on various unrelated “non-obvious subject matter.”8

II. HISTORY OF SOFTWARE PATENTS

One of the most inherent problems with VR-based patents is that they typically fall under two major categories: hardware and software. Although the regulations around hardware patents have been long established, much of the software for VR patents havenot yet been determined due to the complex nature of the intellectual property. This may lead to some complications when comparing software patents to each other as software is inherently intangible and more serves as intellectual property as discussed earlier. This section will discuss precedents set regarding hardware-related VR patents as well assoftware-related VR patents. At the end of this section, an overview of how the Supreme Court currently views VR patents will be discussed, as well as potential ways to ensure company patents fulfill these requirements. As

7 Conditions for patentability; novelty, 35 U.S.C § 102 (2011). 8 Conditions for patentability; non-obvious subject matter, 35 U.S.C § 103 (2011).

17

hardware patents are well established and understood for VR, software patents regarding VR are going to be the focus instead. Two of the most important historical cases regarding programs, in general, are Parker v Flook 1978and Diamond v Diehr 1981. Both of these are considered landmark cases that have established precedents for software patents and in turn software-based VR patents. Additionally, a modern patent will be analyzed to determine how the rulings in prior patent cases are applied.

A. PARKER V FLOOK

Parker v Flook, the earlier of these two cases establishes the limitations of purely software-based patents. In this case, a catalytic converter, a device that helps the refining process of oil by separating pollutants, is in question. Specifically, Dale R. Flook applied for a patent for his software, which was based on a complex mathematical formula that automates the process. Although an impressive mathematical feat, the Burger Court, the ruling court at the time, determined in a 6-3 majority that the whole patent was based on a singular mathematical formula that is simply run through software to seemingly create a unique patent. It was officially ruled that because the claimed invention was absent of some other novel or inventive concept, the algorithm as a principle cannot be transformed into a patentable invention.9

B. DIAMOND V DIEHR

In Diamond v Diehr, a more relevant topic was at hand. The question was “can one patent a machine that transforms materials physically under the control of a programmed computer?” To simplify, the question was asking whether or not hardware controlled by software can be a patentable

9 Parker v. Flook, Oyez, https://www.oyez.org/cases/1977/77-642 (last visited April 22, 2022).

18

invention, which is especially important to VR as it requires both hardware and software patents working in tandem. In this case, James Diehr and Theodore Lutton invented a press that could cure rubber using a heating system that had its temperature automatically measured and used the values from the hardware to continuously calculate cure times and compute the perfect cure time alongwith elapsed time. Previously in Gottschalk v. Benson 1972, a similar case was rejected as the court ruled that combining an unpatentable program with an already patented rubber press was an unpatentable concept. However, in this case, the precedent was overturned. In a 5-4 decision, it was determined that a machine that is controlled by a programmed computer is patentable. Additionally, the case also determined that not all computer programs are unpatentable, opening a plethora of possibilities. This was further solidified thirteen years later when the Court of Appeals ruled that all computer programs are patentable under the requirements of a traditional patent.10

One of the most recent patents in VR is the “3D audio rendering using volumetric audio rendering and scripted audio level-of-detail” patent by investors based in Apple, a modern technology company renowned for their handheld devices.11 The invention in question is essentially a softwaredeveloped audio engine that renders sound for a three-dimensional virtual environment. In this case, the holding from Diamond v Diehr applies, where it was determined that virtually all computer programs are patentable under the conditions of traditional patents. As a result, because this audio engine is completely novel, it was determined that this would be a valid patent and applied to VR. However, this is one of the clearer applications of the rulings. There are far more complex ongoing patent applications that are harder to determine.

10 Diamond v. Diehr, Oyez, https://www.oyez.org/cases/1980/79-1112 (last visited April 22, 2022). 11 U.S Patent No 11,146,905, (filed September 17, 2020).

19

III. CURRENT PATENTS IN VIRTUAL REALITY

This section will discuss the various patent applications that are in the process of being accepted or rejected related to VR. Due to the sheer number of applications in this field, only one major application will be discussed, that being software.

A. THE PORTION OF A DISPLAY SCREEN WITH A SET OF

AVATARS (APPLICATION NO. 29/722,112)

In this application, a portion of the screen is allocated to avatar sets, as the claimed patent entails through the name of the application. In addition, it has specific ornamental designs for avatars that can be applied to various software assets, inclusive of virtual reality and augmented reality.12 The software applies itself to the VR hardware directly, allowing for both hardware and software viewing, hence making it patentable. The mathematical formulas and algorithms behind this software patent application are intertwined with the hardware, leading to the conclusion that this patent application will most likely be accepted.

B. COORDINATING COMMUNICATION LINKS FOR ARTIFICIAL

REALITY (APPLICATION NO. 17/171,843)

This patent application addresses one of the more complex aspects of virtual reality, analyzing joint coordination of communication between two devices/beacons. There are two nodes of communication that displays information such as transmission time. This references the connection between virtual reality devices and servers or other nodes of connection.13 Considering that this application also utilizes the fact that VR requires

12 U.S Patent Application No. 29/722,112, (filed January 20, 2020). 13 U.S Patent Application No. 17/171,843, (filed February 9, 2021).

20

hardware, like the previous patent application, it serves as both a bridged hardware and software patent application. This increases the chances of this claimed patent becoming an actual patent in the near future if considering the invention novel. In all, patents regarding VR and software have certainly made it obvious that the software must either be one of the following: virtually entirely novel in algorithms and mathematical formulas or intertwined with physical hardware to conduct a unique action. In addition to both of these untold rules of software patents, the claimed patent must not have any coverage from previous software patents, as recycling previous algorithms into your own program would be a violation of the novelty of your invention. However, if the software utilizes previous patents and the inventors develop their own unique algorithms that work alongside the previous patents to produce a novel result, the software is patentable. With these conditions in mind, it is now clear that the process of producing a VR patent is far more difficult than at first glance.

CONCLUSION

VR has taken the world by storm and will continue to do so, becoming more prevalent each coming year. As technology continues to gradually develop, various companies in the industry will inevitably file for patents to defend their novel inventions. In turn, the regulations regarding patentability for software and intellectual property will have to be set via precedent cases or changes directly in the U.S. Code, the former being more likely as they already have major decisions established. Additionally, due to the heavy emphasis on software patents for VR, patents on software should focus on the algorithms behind a novel abstract idea instead of just the name of the invention. As is always the case in software, a real-life version of the idea already exists, therefore, it would be impractical to develop a brand-newpatent just for its software application. For example, developing a toaster in real-life is deserving

21

of a patent in its own right, however, making a toaster simulation using software would not necessarily entail the creation of a patent. Instead, courts will have to find a way to differentiate between novel abstract ideas in VR and their algorithms with already existing patents that perform the same action in a different space: real life. As a result of these concerns, VR companies looking to develop software patents should be more vigilant and cautious when applying, as they would need to ensure that their invention properly fulfills the requirements of a patent.

22