Computational law is the branch of legal informatics concerned with the automation of legal reasoning. What distinguishes Computational Law systems from other instances of legal technology is their autonomy, i.e. the ability to answer legal questions without additional input from human legal experts. While there are many possible applications of Computational Law, the primary focus of work in the field today is compliance management, i.e. the development and deployment of computer systems capable of assessing, facilitating, or enforcing compliance with rules and regulations. Some systems of this sort already exist. TurboTax is a good example. And the potential is particularly significant now due to recent technological advances – including the prevalence of the Internet in human interaction and the proliferation of embedded computer systems (such as smart phones, self-driving cars, and robots). There are also applications that do not involve governmental laws. The regulations can just as well be the terms of contracts (e.g. delivery schedules, insurance covenants, real estate transactions, financial agreements). They can be the policies of corporations (e.g. constraints on travel, expenditure reporting, pricing rules). They can even be the rules of games (embodied in computer game playing systems). == History == Speculation about potential benefits to legal practice through applying methods from computational science and AI research to automate parts of the law date back at least to the middle 1940s. Further, AI and law and computational law do not seem easily separable, as perhaps most of AI research focusing on the law and its automation appears to utilize computational methods. The forms that speculation took are multiple and not all related in ways to readily show closeness to one another. This history will sketch them as they were, attempting to show relationships where they can be found to have existed. By 1949, a minor academic field aiming to incorporate electronic and computational methods to legal problems had been founded by American legal scholars, called jurimetrics. Though broadly said to be concerned with the application of the "methods of science" to the law, these methods were actually of a quite specifically defined scope. Jurimetrics was to be "concerned with such matters as the quantitative analysis of judicial behavior, the application of communication and information theory to legal expression, the use of mathematical logic in law, the retrieval of legal data by electronic and mechanical means, and the formulation of a calculus of legal predictability". These interests led in 1959 to the founding a journal, Modern Uses of Logic in Law, as a forum wherein articles would be published about the applications of techniques such as mathematical logic, engineering, statistics, etc. to the legal study and development. In 1966, this Journal was renamed as Jurimetrics. Today, however, the journal and meaning of jurimetrics seems to have broadened far beyond what would fit under the areas of applications of computers and computational methods to law. Today the journal not only publishes articles on such practices as found in computational law, but has broadened jurimetrical concerns to mean also things like the use of social science in law or the "policy implications [of] and legislative and administrative control of science". Independently in 1958, at the Conference for the Mechanization of Thought held at the National Physical Laboratory in Teddington, Middlesex, UK, the French jurist Lucien Mehl presented a paper both on the benefits of using computational methods for law and on the potential means to use such methods to automate law for a discussion that included AI luminaries like Marvin Minsky. Mehl believed that the law could by automated by two basic distinct, though not wholly separable, types of machine. These were the "documentary or information machine", which would provide the legal researcher quick access to relevant case precedents and legal scholarship, and the "consultation machine", which would be "capable of answering any question put to it over a vast field of law". The latter type of machine would be able to basically do much of a lawyer's job by simply giving the "exact answer to a [legal] problem put to it". By 1970, Mehl's first type of machine, one that would be able to retrieve information, had been accomplished but there seems to have been little consideration of further fruitful intersections between AI and legal research. There were, however, still hopes that computers could model the lawyer's thought processes through computational methods and then apply that capacity to solve legal problems, thus automating and improving legal services via increased efficiency as well as shedding light on the nature of legal reasoning. By the late 1970s, computer science and the affordability of computer technology had progressed enough that the retrieval of "legal data by electronic and mechanical means" had been achieved by machines fitting Mehl's first type and were in common use in American law firms. During this time, research focused on improving the goals of the early 1970s occurred, with programs like Taxman being worked on in order to both bring useful computer technology into the law as practical aids and to help specify the exact nature of legal concepts. Nonetheless, progress on the second type of machine, one that would more fully automate the law, remained relatively inert. Research into machines that could answer questions in the way that Mehl's consultation machine would picked up somewhat in the late 1970s and 1980s. A 1979 convention in Swansea, Wales marked the first international effort solely to focus upon applying artificial intelligence research to legal problems in order to "consider how computers can be used to discover and apply the legal norms embedded within the written sources of the law". Considerable progress on the development of the second type of machine was made in the following decade, with the development of a variety of expert systems. According to Thorne McCarty, "these systems all have the following characteristics: They do backward chaining inference from a specified goal; they ask questions to elicit information from the user; and they produce a suggested answer along with a trace of the supporting legal rules." According to Prakken and Sartor the representation of the British Nationality Act as a logic program, which introduced this approach, was "hugely influential for the development of computational representations of legislation, showing how logic programming enables intuitively appealing representations that can be directly deployed to generate automatic inferences". In 2021, this work received the Inaugural CodeX Prize as "one of the first and best-known works in computational law, and one of the most widely cited papers in the field." In a 1988 review of Anne Gardner's book An Artificial Intelligence Approach to Legal Reasoning (1987), the Harvard academic legal scholar and computer scientist Edwina Rissland wrote that "She plays, in part, the role of pioneer; artificial intelligence ("AI") techniques have not yet been widely applied to perform legal tasks. Therefore, Gardner, and this review, first describe and define the field, then demonstrate a working model in the domain of contract offer and acceptance." Eight years after the Swansea conference had passed, and still AI and law researchers merely trying to delineate the field could be described by their own kind as "pioneer[s]". In the 1990s and early 2000s more progress occurred. Computational research generated insights for law. The First International Conference on AI and the Law occurred in 1987, but it is in the 1990s and 2000s that the biannual conference began to build up steam and to delve more deeply into the issues involved with work intersecting computational methods, AI, and law. Classes began to be taught to undergraduates on the uses of computational methods to automating, understanding, and obeying the law. Further, by 2005, a team largely composed of Stanford computer scientists from the Stanford Logic group had devoted themselves to studying the uses of computational techniques to the law. Computational methods in fact advanced enough that members of the legal profession began in the 2000s to both analyze, predict and worry about the potential future of computational law and a new academic field of computational legal studies seems to be now well established. As insight into what such scholars see in the law's future due in part to computational law, here is quote from a recent conference about the "New Normal" for the legal profession: "Over the last 5 years, in the fallout of the Great Recession, the legal profession has entered the era of the New Normal. Notably, a series of forces related to technological change, globalization, and the pressure to do more with less (in both corpo
Natural language understanding
Natural language understanding (NLU) or natural language interpretation (NLI) is a subset of natural language processing in artificial intelligence that deals with machine reading comprehension. NLU has been considered an AI-hard problem. There is considerable commercial interest in the field because of its application to automated reasoning, machine translation, question answering, news-gathering, text categorization, voice-activation, archiving, and large-scale content analysis. == History == The program STUDENT, written in 1964 by Daniel Bobrow for his PhD dissertation at MIT, is one of the earliest known attempts at NLU by a computer. Eight years after John McCarthy coined the term artificial intelligence, Bobrow's dissertation (titled Natural Language Input for a Computer Problem Solving System) showed how a computer could understand simple natural language input to solve algebra word problems. A year later, in 1965, Joseph Weizenbaum at MIT wrote ELIZA, an interactive program that carried on a dialogue in English on any topic, the most popular being psychotherapy. ELIZA worked by simple parsing and substitution of key words into canned phrases and Weizenbaum sidestepped the problem of giving the program a database of real-world knowledge or a rich lexicon. Yet ELIZA gained surprising popularity as a toy project and can be seen as a very early precursor to current commercial systems such as those used by Ask.com. In 1969, Roger Schank at Stanford University introduced the conceptual dependency theory for NLU. This model, partially influenced by the work of Sydney Lamb, was extensively used by Schank's students at Yale University, such as Robert Wilensky, Wendy Lehnert, and Janet Kolodner. In 1970, William A. Woods introduced the augmented transition network (ATN) to represent natural language input. Instead of phrase structure rules ATNs used an equivalent set of finite-state automata that were called recursively. ATNs and their more general format called "generalized ATNs" continued to be used for a number of years. In 1971, Terry Winograd finished writing SHRDLU for his PhD thesis at MIT. SHRDLU could understand simple English sentences in a restricted world of children's blocks to direct a robotic arm to move items. The successful demonstration of SHRDLU provided significant momentum for continued research in the field. Winograd continued to be a major influence in the field with the publication of his book Language as a Cognitive Process. At Stanford, Winograd would later advise Larry Page, who co-founded Google. In the 1970s and 1980s, the natural language processing group at SRI International continued research and development in the field. A number of commercial efforts based on the research were undertaken, e.g., in 1982 Gary Hendrix formed Symantec Corporation originally as a company for developing a natural language interface for database queries on personal computers. However, with the advent of mouse-driven graphical user interfaces, Symantec changed direction. A number of other commercial efforts were started around the same time, e.g., Larry R. Harris at the Artificial Intelligence Corporation and Roger Schank and his students at Cognitive Systems Corp. In 1983, Michael Dyer developed the BORIS system at Yale which bore similarities to the work of Roger Schank and W. G. Lehnert. The third millennium saw the introduction of systems using machine learning for text classification, such as the IBM Watson. However, experts debate how much "understanding" such systems demonstrate: e.g., according to John Searle, Watson did not even understand the questions. John Ball, cognitive scientist and inventor of the Patom Theory, supports this assessment. Natural language processing has made inroads for applications to support human productivity in service and e-commerce, but this has largely been made possible by narrowing the scope of the application. There are thousands of ways to request something in a human language that still defies conventional natural language processing. According to Wibe Wagemans, "To have a meaningful conversation with machines is only possible when we match every word to the correct meaning based on the meanings of the other words in the sentence – just like a 3-year-old does without guesswork." == Scope and context == The umbrella term "natural language understanding" can be applied to a diverse set of computer applications, ranging from small, relatively simple tasks such as short commands issued to robots, to highly complex endeavors such as the full comprehension of newspaper articles or poetry passages. Many real-world applications fall between the two extremes, for instance text classification for the automatic analysis of emails and their routing to a suitable department in a corporation does not require an in-depth understanding of the text, but needs to deal with a much larger vocabulary and more diverse syntax than the management of simple queries to database tables with fixed schemata. Throughout the years various attempts at processing natural language or English-like sentences presented to computers have taken place at varying degrees of complexity. Some attempts have not resulted in systems with deep understanding, but have helped overall system usability. For example, Wayne Ratliff originally developed the Vulcan program with an English-like syntax to mimic the English speaking computer in Star Trek. Vulcan later became the dBase system whose easy-to-use syntax effectively launched the personal computer database industry. Systems with an easy-to-use or English-like syntax are, however, quite distinct from systems that use a rich lexicon and include an internal representation (often as first order logic) of the semantics of natural language sentences. Hence the breadth and depth of "understanding" aimed at by a system determine both the complexity of the system (and the implied challenges) and the types of applications it can deal with. The "breadth" of a system is measured by the sizes of its vocabulary and grammar. The "depth" is measured by the degree to which its understanding approximates that of a fluent native speaker. At the narrowest and shallowest, English-like command interpreters require minimal complexity, but have a small range of applications. Narrow but deep systems explore and model mechanisms of understanding, but they still have limited application. Systems that attempt to understand the contents of a document such as a news release beyond simple keyword matching and to judge its suitability for a user are broader and require significant complexity, but they are still somewhat shallow. Systems that are both very broad and very deep are beyond the current state of the art. == Components and architecture == Regardless of the approach used, most NLU systems share some common components. The system needs a lexicon of the language and a parser and grammar rules to break sentences into an internal representation. The construction of a rich lexicon with a suitable ontology requires significant effort, e.g., the Wordnet lexicon required many person-years of effort. The system also needs theory from semantics to guide the comprehension. The interpretation capabilities of a language-understanding system depend on the semantic theory it uses. Competing semantic theories of language have specific trade-offs in their suitability as the basis of computer-automated semantic interpretation. These range from naive semantics or stochastic semantic analysis to the use of pragmatics to derive meaning from context. Semantic parsers convert natural-language texts into formal meaning representations. Advanced applications of NLU also attempt to incorporate logical inference within their framework. This is generally achieved by mapping the derived meaning into a set of assertions in predicate logic, then using logical deduction to arrive at conclusions. Therefore, systems based on functional languages such as Lisp need to include a subsystem to represent logical assertions, while logic-oriented systems such as those using the language Prolog generally rely on an extension of the built-in logical representation framework. The management of context in NLU can present special challenges. A large variety of examples and counter examples have resulted in multiple approaches to the formal modeling of context, each with specific strengths and weaknesses.
AI nationalism
AI nationalism is the idea that nations should develop and control their own artificial intelligence technologies to advance their own interests and ensure technological sovereignty. This concept is gaining traction globally, leading countries to implement new laws, form strategic alliances, and invest significantly in domestic AI capabilities. == Global trends and national strategies == In 2018, British technology investor Ian Hogarth published an influential essay titled AI Nationalism. He argued that as AI gains more power and its economic and military significance expands, governments will take measures to bolster their own domestic AI industries, and predicted that the advancement of machine learning systems would lead to what he termed "AI nationalism." He anticipated that this rise in AI would accelerate a global arms race, resulting in more closed economies, restrictions on foreign acquisitions, and limitations on the movement of talent. Hogarth predicted that AI policy would become a central focus of government agendas. He also criticized Britain’s approach to AI strategy, citing the sale of London-based DeepMind—one of the leading AI laboratories, acquired by Google for a relatively modest £400 million in 2014—as a significant misstep. AI nationalism is chiefly reflected in the escalating rhetoric of an artificial intelligence arms race, portraying AI development as a zero-sum game where the winner gains significant economic, political, and military advantages. This mindset, as highlighted in a 2017 Pentagon report, warns that sharing AI technology could erode technological supremacy and enhance rivals' capabilities. The winner-takes-all mentality of AI nationalism poses risks including unsafe AI development, increased geopolitical tension, and potential military aggression (such as cyberattacks or targeting AI professionals). Several countries, including Canada, France, and India, have formulated national strategies to advance their positions in AI. In the United States, a leading player in the global AI arena, trade policies have been enacted to restrict China's access to critical microchips, reflecting a strategic effort to maintain a technological edge. The United States’ National Security Commission on Artificial Intelligence (NSCAI) frames AI development as a critical aspect of a broader technology competition crucial for national success. It emphasizes the need to outpace China in AI to maintain strategic advantage, reflecting AI nationalism by linking geopolitical power directly to advancements in AI. France has seen notable governmental support for local AI startups, particularly those specializing in language technologies that cater to French and other non-English languages. In Saudi Arabia, Crown Prince Mohammed bin Salman is investing billions in AI research and development. The country has actively collaborated with major technology firms such as Amazon, IBM, and Microsoft to establish itself as a prominent AI hub. == Historical and cultural context == AI nationalism is seen as deeply connected to historical racism and imperialism. It is viewed not merely as a technological competition but as a contest over racial and civilizational superiority. Historically, technological achievements were often used to justify colonialism and racial hierarchies, with Western societies perceiving their advancements as evidence of superiority. In the context of AI, this historical context continues to shape views on intelligence and development. Some argue that AI nationalism reinforces the idea of fundamental civilizational divides, especially between the Western world and China. This perspective often frames China's progress in AI as a direct challenge to Western values, presenting the AI competition as a struggle over values. AI nationalism is said to draw from long-standing anti-Asian stereotypes, such as the "Yellow Peril," which portray Asian nations as threats to Western civilization. This viewpoint links Asian technological advances with dehumanization and artificiality, reflecting persistent anxieties about China's growing role in the global tech landscape. == Implications == AI nationalism is seen as a component of a broader trend towards the fragmentation of the internet, where digital services are increasingly influenced by local regulations and national interests. This shift is creating a new technological landscape in which the impact of artificial intelligence on individuals' lives can vary significantly depending on their geographic location. J. Paul Goode argues that AI nationalism may exacerbate existing societal divisions by promoting the development of systems that embed cultural biases, thereby privileging certain groups while disadvantaging others.
Concurrent MetateM
Concurrent MetateM is a multi-agent language in which each agent is programmed using a set of (augmented) temporal logic specifications of the behaviour it should exhibit. These specifications are executed directly to generate the behaviour of the agent. As a result, there is no risk of invalidating the logic as with systems where logical specification must first be translated to a lower-level implementation. The root of the MetateM concept is Gabbay's separation theorem; any arbitrary temporal logic formula can be rewritten in a logically equivalent past → future form. Execution proceeds by a process of continually matching rules against a history, and firing those rules when antecedents are satisfied. Any instantiated future-time consequents become commitments which must subsequently be satisfied, iteratively generating a model for the formula made up of the program rules. == Temporal Connectives == The Temporal Connectives of Concurrent MetateM can divided into two categories, as follows: Strict past time connectives: '●' (weak last), '◎' (strong last), '◆' (was), '■' (heretofore), 'S' (since), and 'Z' (zince, or weak since). Present and future time connectives: '◯' (next), '◇' (sometime), '□' (always), 'U' (until), and 'W' (unless). The connectives {◎,●,◆,■,◯,◇,□} are unary; the remainder are binary. === Strict past time connectives === ==== Weak last ==== ●ρ is satisfied now if ρ was true in the previous time. If ●ρ is interpreted at the beginning of time, it is satisfied despite there being no actual previous time. Hence "weak" last. ==== Strong last ==== ◎ρ is satisfied now if ρ was true in the previous time. If ◎ρ is interpreted at the beginning of time, it is not satisfied because there is no actual previous time. Hence "strong" last. ==== Was ==== ◆ρ is satisfied now if ρ was true in any previous moment in time. ==== Heretofore ==== ■ρ is satisfied now if ρ was true in every previous moment in time. ==== Since ==== ρSψ is satisfied now if ψ is true at any previous moment and ρ is true at every moment after that moment. ==== Zince, or weak since ==== ρZψ is satisfied now if (ψ is true at any previous moment and ρ is true at every moment after that moment) OR ψ has not happened in the past. === Present and future time connectives === ==== Next ==== ◯ρ is satisfied now if ρ is true in the next moment in time. ==== Sometime ==== ◇ρ is satisfied now if ρ is true now or in any future moment in time. ==== Always ==== □ρ is satisfied now if ρ is true now and in every future moment in time. ==== Until ==== ρUψ is satisfied now if ψ is true at any future moment and ρ is true at every moment prior. ==== Unless ==== ρWψ is satisfied now if (ψ is true at any future moment and ρ is true at every moment prior) OR ψ does not happen in the future.
Matchbox Educable Noughts and Crosses Engine
The Matchbox Educable Noughts and Crosses Engine (sometimes called the Machine Educable Noughts and Crosses Engine or MENACE) was a mechanical computer made from 304 matchboxes designed and built by artificial intelligence researcher Donald Michie and his colleague Roger Chambers, in 1961. It was designed to play human opponents in games of noughts and crosses (tic-tac-toe) by returning a move for any given state of play and to refine its strategy through reinforcement learning. This was one of the first types of artificial intelligence. Michie and Chambers did not have immediate access to a computer; they worked around this by building the engine out of matchboxes. The matchboxes they used each represented a single possible layout of a noughts and crosses grid. When the computer first played, it would randomly choose moves based on the current layout. As it played more games, through a reinforcement loop, it disqualified strategies that led to losing games, and supplemented strategies that led to winning games. Michie held a tournament against MENACE in 1961, wherein he experimented with different openings. Following MENACE's maiden tournament against Michie, it demonstrated successful artificial intelligence in its strategy. Michie's essays on MENACE's weight initialisation and the BOXES algorithm used by MENACE became popular in the field of computer science research. Michie was honoured for his contribution to machine learning research, and was twice commissioned to program a MENACE simulation on an actual computer. == Origin == Donald Michie (1923–2007) had been on the team decrypting the German Tunny Code during World War II. Fifteen years later, he wanted to further display his mathematical and computational prowess with an early convolutional neural network. Since computer equipment was not obtainable for such uses, and Michie did not have a computer readily available, he decided to display and demonstrate artificial intelligence in a more esoteric format and constructed a functional mechanical computer out of matchboxes and beads. MENACE was constructed as the result of a bet with a computer science colleague who postulated that such a machine was impossible. Michie undertook the task of collecting and defining each matchbox as a "fun project", later turned into a demonstration tool. Michie completed his essay on MENACE in 1963, "Experiments on the mechanization of game-learning", as well as his essay on the BOXES Algorithm, written with R. A. Chambers and had built up an AI research unit in Hope Park Square, Edinburgh, Scotland. MENACE learned by playing successive matches of noughts and crosses. Each time, it would eliminate a losing strategy by the human player confiscating the beads that corresponded to each move. It reinforced winning strategies by making the moves more likely, by supplying extra beads. This was one of the earliest versions of the Reinforcement Loop, the schematic algorithm of looping the algorithm, dropping unsuccessful strategies until only the winning ones remain. This model starts as completely random, and gradually learns. == Composition == MENACE was made from 304 matchboxes glued together in an arrangement similar to a chest of drawers. Each box had a code number, which was keyed into a chart. This chart had drawings of tic-tac-toe game grids with various configurations of X, O, and empty squares, corresponding to all possible permutations a game could go through as it progressed. After removing duplicate arrangements (ones that were simply rotations or mirror images of other configurations), MENACE used 304 permutations in its chart and thus that many matchboxes. Each individual matchbox tray contained a collection of coloured beads. Each colour represented a move on a square on the game grid, and so matchboxes with arrangements where positions on the grid were already taken would not have beads for that position. Additionally, at the front of the tray were two extra pieces of card in a "V" shape, the point of the "V" pointing at the front of the matchbox. Michie and his artificial intelligence team called MENACE's algorithm "Boxes", after the apparatus used for the machine. The first stage "Boxes" operated in five phases, each setting a definition and a precedent for the rules of the algorithm in relation to the game. == Operation == MENACE played first, as O, since all matchboxes represented permutations only relevant to the "X" player. To retrieve MENACE's choice of move, the opponent or operator located the matchbox that matched the current game state, or a rotation or mirror image of it. For example, at the start of a game, this would be the matchbox for an empty grid. The tray would be removed and lightly shaken so as to move the beads around. Then, the bead that had rolled into the point of the "V" shape at the front of the tray was the move MENACE had chosen to make. Its colour was then used as the position to play on, and, after accounting for any rotations or flips needed based on the chosen matchbox configuration's relation to the current grid, the O would be placed on that square. Then the player performed their move, the new state was located, a new move selected, and so on, until the game was finished. When the game had finished, the human player observed the game's outcome. As a game was played, each matchbox that was used for MENACE's turn had its tray returned to it ajar, and the bead used kept aside, so that MENACE's choice of moves and the game states they belonged to were recorded. Michie described his reinforcement system with "reward" and "punishment". Once the game was finished, if MENACE had won, it would then receive a "reward" for its victory. The removed beads showed the sequence of the winning moves. These were returned to their respective trays, easily identifiable since they were slightly open, as well as three bonus beads of the same colour. In this way, in future games MENACE would become more likely to repeat those winning moves, reinforcing winning strategies. If it lost, the removed beads were not returned, "punishing" MENACE, and meaning that in future it would be less likely, and eventually incapable if that colour of bead became absent, to repeat the moves that cause a loss. If the game was a draw, one additional bead was added to each box. == Results in practice == === Optimal strategy === Noughts and crosses has a well-known optimal strategy. A player must place their symbol in a way that blocks the other player from achieving any rows while simultaneously making a row themself. However, if both players use this strategy, the game always ends in a draw. If the human player is familiar with the optimal strategy, and MENACE can quickly learn it, then the games will eventually only end in draws. The likelihood of the computer winning increases quickly when the computer plays against a random-playing opponent. When playing against a player using optimal strategy, the odds of a draw grow to 100%. In Donald Michie's official tournament against MENACE in 1961 he used optimal strategy, and he and the computer began to draw consistently after twenty games. Michie's tournament had the following milestones: Michie began by consistently opening with "Variant 0", the middle square. At 15 games, MENACE abandoned all non-corner openings. At just over 20, Michie switched to consistently using "Variant 1", the bottom-right square. At 60, he returned to Variant 0. As he neared 80 games, he moved to "Variant 2", the top-middle. At 110, he switched to "Variant 3", the top right. At 135, he switched to "Variant 4", middle-right. At 190, he returned to Variant 1, and at 210, he returned to Variant 0. The trend in changes of beads in the "2" boxes runs: === Correlation === Depending on the strategy employed by the human player, MENACE produces a different trend on scatter graphs of wins. Using a random turn from the human player results in an almost-perfect positive trend. Playing the optimal strategy returns a slightly slower increase. The reinforcement does not create a perfect standard of wins; the algorithm will draw random uncertain conclusions each time. After the j-th round, the correlation of near-perfect play runs: 1 − D D − D ( j + 2 ) ∑ i = 0 j D ( j i + 1 ) V i {\displaystyle {1-D \over D-D^{(j+2)}}\sum _{i=0}^{j}D^{(ji+1)}V_{i}} Where Vi is the outcome (+1 is win, 0 is draw and -1 is loss) and D is the decay factor (average of past values of wins and losses). Below, Mn is the multiplier for the n-th round of the game. == Legacy == Donald Michie's MENACE proved that a computer could learn from failure and success to become good at a task. It used what would become core principles within the field of machine learning before they had been properly theorised. For example, the combination of how MENACE starts with equal numbers of types of beads in each matchbox, and how these are then selected at random, creates a learning behaviour similar to weight initialisation
PhyCV
PhyCV is the first computer vision library which utilizes algorithms directly derived from the equations of physics governing physical phenomena. The algorithms appearing in the first release emulate the propagation of light through a physical medium with natural and engineered diffractive properties followed by coherent detection. Unlike traditional algorithms that are a sequence of hand-crafted empirical rules, physics-inspired algorithms leverage physical laws of nature as blueprints. In addition, these algorithms can, in principle, be implemented in real physical devices for fast and efficient computation in the form of analog computing. Currently PhyCV has three algorithms, Phase-Stretch Transform (PST) and Phase-Stretch Adaptive Gradient-Field Extractor (PAGE), and Vision Enhancement via Virtual diffraction and coherent Detection (VEViD). All algorithms have CPU and GPU versions. PhyCV is now available on GitHub and can be installed from pip. == History == Algorithms in PhyCV are inspired by the physics of the photonic time stretch (a hardware technique for ultrafast and single-shot data acquisition). PST is an edge detection algorithm that was open-sourced in 2016 and has 800+ stars and 200+ forks on GitHub. PAGE is a directional edge detection algorithm that was open-sourced in February, 2022. PhyCV was originally developed and open-sourced by Jalali-Lab @ UCLA in May 2022. In the initial release of PhyCV, the original open-sourced code of PST and PAGE is significantly refactored and improved to be modular, more efficient, GPU-accelerated and object-oriented. VEViD is a low-light and color enhancement algorithm that was added to PhyCV in November 2022. == Background == === Phase-Stretch Transform (PST) === Phase-Stretch Transform (PST) is a computationally efficient edge and texture detection algorithm with exceptional performance in visually impaired images. The algorithm transforms the image by emulating propagation of light through a device with engineered diffractive property followed by coherent detection. It has been applied in improving the resolution of MRI image, extracting blood vessels in retina images, dolphin identification, and waste water treatment, single molecule biological imaging, and classification of UAV using micro Doppler imaging. === Phase-Stretch Adaptive Gradient-Field Extractor (PAGE) === Phase-Stretch Adaptive Gradient-Field Extractor (PAGE) is a physics-inspired algorithm for detecting edges and their orientations in digital images at various scales. The algorithm is based on the diffraction equations of optics. Metaphorically speaking, PAGE emulates the physics of birefringent (orientation-dependent) diffractive propagation through a physical device with a specific diffractive structure. The propagation converts a real-valued image into a complex function. Related information is contained in the real and imaginary components of the output. The output represents the phase of the complex function. === Vision Enhancement via Virtual diffraction and coherent Detection (VEViD) === Vision Enhancement via Virtual diffraction and coherent Detection (VEViD) an efficient and interpretable low-light and color enhancement algorithm that reimagines a digital image as a spatially varying metaphoric light field and then subjects the field to the physical processes akin to diffraction and coherent detection. The term “Virtual” captures the deviation from the physical world. The light field is pixelated and the propagation imparts a phase with an arbitrary dependence on frequency which can be different from the quadratic behavior of physical diffraction. VEViD can be further accelerated through mathematical approximations that reduce the computation time without appreciable sacrifice in image quality. A closed-form approximation for VEViD which we call VEViD-lite can achieve up to 200 FPS for 4K video enhancement. == PhyCV on the Edge == Featuring low-dimensionality and high-efficiency, PhyCV is ideal for edge computing applications. In this section, we demonstrate running PhyCV on NVIDIA Jetson Nano in real-time. === NVIDIA Jetson Nano Developer Kit === NVIDIA Jetson Nano Developer Kit is a small- sized and power-efficient platform for edge computing applications. It is equipped with an NVIDIA Maxwell architecture GPU with 128 CUDA cores, a quad-core ARM Cortex-A57 CPU, 4GB 64-bit LPDDR4 RAM, and supports video encoding and decoding up to 4K resolution. Jetson Nano also offers a variety of interfaces for connectivity and expansion, making it ideal for a wide range of AI and IoT applications. In our setup, we connect a USB camera to the Jetson Nano to acquire videos and demonstrate using PhyCV to process the videos in real-time. === Real-time PhyCV on Jetson Nano === We use the Jetson Nano (4GB) with NVIDIA JetPack SDK version 4.6.1, which comes with pre- installed Python 3.6, CUDA 10.2, and OpenCV 4.1.1. We further install PyTorch 1.10 to enable the GPU accelerated PhyCV. We demonstrate the results and metrics of running PhyCV on Jetson Nano in real-time for edge detection and low-light enhancement tasks. For 480p videos, both operations achieve beyond 38 FPS, which is sufficient for most cameras that capture videos at 30 FPS. For 720p videos, PhyCV low-light enhancement can operate at 24 FPS and PhyCV edge detection can operate at 17 FPS. == Highlights == === Modular Code Architecture === The code in PhyCV has a modular design which faithfully follows the physical process from which the algorithm was originated. Both PST and PAGE modules in the PhyCV library emulate the propagation of the input signal (original digital image) through a device with engineered diffractive property followed by coherent (phase) detection. The dispersive propagation applies a phase kernel to the frequency domain of the original image. This process has three steps in general, loading the image, initializing the kernel and applying the kernel. In the implementation of PhyCV, each algorithm is represented as a class in Python and each class has methods that simulate the steps described above. The modular code architecture follows the physics behind the algorithm. Please refer to the source code on GitHub for more details. === GPU Acceleration === PhyCV supports GPU acceleration. The GPU versions of PST and PAGE are built on PyTorch accelerated by the CUDA toolkit. The acceleration is beneficial for applying the algorithms in real-time image video processing and other deep learning tasks. The running time per frame of PhyCV algorithms on CPU (Intel i9-9900K) and GPU (NVIDIA TITAN RTX) for videos at different resolutions are shown below. Note that the PhyCV low-light enhancement operates in the HSV color space, so the running time also includes RGB to HSV conversion. However, for all running times using GPUs, we ignore the time of moving data from CPUs to GPUs and count the algorithm operation time only. == Installation and Examples == Please refer to the GitHub README file for a detailed technical documentation. == Current Limitations == === I/O (Input/Output) Bottleneck for Real-time Video Processing === When dealing with real-time video streams from cameras, the frames are captured and buffered in CPU and have to be moved to GPU to run the GPU-accelerated PhyCV algorithms. This process is time-consuming and it is a common bottleneck for real-time video-processing algorithms. === Lack of Parameter Adaptivity for Different Images === Currently, the parameters of PhyCV algorithms have to be manually tuned for different images. Although a set of pre-selected parameters work relatively well for a wide range of images, the lack of parameter adaptivity for different images remains a limitation for now.
Tensor (machine learning)
In machine learning, the term tensor informally refers to two different concepts: (i) a way of organizing data and (ii) a multilinear (tensor) transformation. Data may be organized in a multidimensional array (M-way array), informally referred to as a "data tensor"; however, in the strict mathematical sense, a tensor is a multilinear mapping over a set of domain vector spaces to a range vector space. Observations, such as images, movies, volumes, sounds, and relationships among words and concepts, stored in an M-way array ("data tensor"), may be analyzed either by artificial neural networks or tensor methods. Tensor decomposition factors data tensors into smaller tensors. Operations on data tensors can be expressed in terms of matrix multiplication and the Kronecker product. The computation of gradients, a crucial aspect of backpropagation, can be performed using software libraries such as PyTorch and TensorFlow. Computations are often performed on graphics processing units (GPUs) using CUDA, and on dedicated hardware such as Google's Tensor Processing Unit or Nvidia's Tensor core. These developments have greatly accelerated neural network architectures, and increased the size and complexity of models that can be trained. == History == A tensor is by definition a multilinear map. In mathematics, this may express a multilinear relationship between sets of algebraic objects. In physics, tensor fields, considered as tensors at each point in space, are useful in expressing mechanics such as stress or elasticity. In machine learning, the exact use of tensors depends on the statistical approach being used. In 2001, the field of signal processing and statistics were making use of tensor methods. Pierre Comon surveys the early adoption of tensor methods in the fields of telecommunications, radio surveillance, chemometrics and sensor processing. Linear tensor rank methods (such as, Parafac/CANDECOMP) analyzed M-way arrays ("data tensors") composed of higher order statistics that were employed in blind source separation problems to compute a linear model of the data. He noted several early limitations in determining the tensor rank and efficient tensor rank decomposition. In the early 2000s, multilinear tensor methods crossed over into computer vision, computer graphics and machine learning with papers by Vasilescu or in collaboration with Terzopoulos, such as Human Motion Signatures, TensorFaces TensorTextures and Multilinear Projection. Multilinear algebra, the algebra of higher-order tensors, is a suitable and transparent framework for analyzing the multifactor structure of an ensemble of observations and for addressing the difficult problem of disentangling the causal factors based on second order or higher order statistics associated with each causal factor. Tensor (multilinear) factor analysis disentangles and reduces the influence of different causal factors with multilinear subspace learning. When treating an image or a video as a 2- or 3-way array, i.e., "data matrix/tensor", tensor methods reduce spatial or time redundancies as demonstrated by Wang and Ahuja. Yoshua Bengio, Geoff Hinton and their collaborators briefly discuss the relationship between deep neural networks and tensor factor analysis beyond the use of M-way arrays ("data tensors") as inputs. One of the early uses of tensors for neural networks appeared in natural language processing. A single word can be expressed as a vector via Word2vec. Thus a relationship between two words can be encoded in a matrix. However, for more complex relationships such as subject-object-verb, it is necessary to build higher-dimensional networks. In 2009, the work of Sutskever introduced Bayesian Clustered Tensor Factorization to model relational concepts while reducing the parameter space. From 2014 to 2015, tensor methods become more common in convolutional neural networks (CNNs). Tensor methods organize neural network weights in a "data tensor", analyze and reduce the number of neural network weights. Lebedev et al. accelerated CNN networks for character classification (the recognition of letters and digits in images) by using 4D kernel tensors. == Definition == Let F {\displaystyle \mathbb {F} } be a field (such as the real numbers R {\displaystyle \mathbb {R} } or the complex numbers C {\displaystyle \mathbb {C} } ). A tensor T ∈ F I 1 × I 2 × … × I C {\displaystyle {\mathcal {T}}\in {\mathbb {F} }^{I_{1}\times I_{2}\times \ldots \times I_{C}}} is a multilinear transformation from a set of domain vector spaces to a range vector space: T : { F I 1 × F I 2 × … F I C } ↦ F I 0 {\displaystyle {\mathcal {T}}:\{{\mathbb {F} }^{I_{1}}\times {\mathbb {F} }^{I_{2}}\times \ldots {\mathbb {F} }^{I_{C}}\}\mapsto {\mathbb {F} }^{I_{0}}} Here, C {\displaystyle C} and I 0 , I 1 , … , I C {\displaystyle I_{0},I_{1},\ldots ,I_{C}} are positive integers, and ( C + 1 ) {\displaystyle (C+1)} is the number of modes of a tensor (also known as the number of ways of a multi-way array). The dimensionality of mode c {\displaystyle c} is I c {\displaystyle I_{c}} , for 0 ≤ c ≤ C {\displaystyle 0\leq c\leq C} . In statistics and machine learning, an image is vectorized when viewed as a single observation, and a collection of vectorized images is organized as a "data tensor". For example, a set of facial images { d i p , i e , i l , i v ∈ R I X } {\displaystyle \{{\mathbb {d} }_{i_{p},i_{e},i_{l},i_{v}}\in {\mathbb {R} }^{I_{X}}\}} with I X {\displaystyle I_{X}} pixels that are the consequences of multiple causal factors, such as a facial geometry i p ( 1 ≤ i p ≤ I P ) {\displaystyle i_{p}(1\leq i_{p}\leq I_{P})} , an expression i e ( 1 ≤ i e ≤ I E ) {\displaystyle i_{e}(1\leq i_{e}\leq I_{E})} , an illumination condition i l ( 1 ≤ i l ≤ I L ) {\displaystyle i_{l}(1\leq i_{l}\leq I_{L})} , and a viewing condition i v ( 1 ≤ i v ≤ I V ) {\displaystyle i_{v}(1\leq i_{v}\leq I_{V})} may be organized into a data tensor (ie. multiway array) D ∈ R I X × I P × I E × I L × V {\displaystyle {\mathcal {D}}\in {\mathbb {R} }^{I_{X}\times I_{P}\times I_{E}\times I_{L}\times V}} where I P {\displaystyle I_{P}} are the total number of facial geometries, I E {\displaystyle I_{E}} are the total number of expressions, I L {\displaystyle I_{L}} are the total number of illumination conditions, and I V {\displaystyle I_{V}} are the total number of viewing conditions. Tensor factorizations methods such as TensorFaces and multilinear (tensor) independent component analysis factorizes the data tensor into a set of vector spaces that span the causal factor representations, where an image is the result of tensor transformation T {\displaystyle {\mathcal {T}}} that maps a set of causal factor representations to the pixel space. Another approach to using tensors in machine learning is to embed various data types directly. For example, a grayscale image, commonly represented as a discrete 2-way array D ∈ R I R X × I C X {\displaystyle {\mathbf {D} }\in {\mathbb {R} }^{I_{RX}\times I_{CX}}} with dimensionality I R X × I C X {\displaystyle I_{RX}\times I_{CX}} where I R X {\displaystyle I_{RX}} are the number of rows and I C X {\displaystyle I_{CX}} are the number of columns. When an image is treated as 2-way array or 2nd order tensor (i.e. as a collection of column/row observations), tensor factorization methods compute the image column space, the image row space and the normalized PCA coefficients or the ICA coefficients. Similarly, a color image with RGB channels, D ∈ R N × M × 3 . {\displaystyle {\mathcal {D}}\in \mathbb {R} ^{N\times M\times 3}.} may be viewed as a 3rd order data tensor or 3-way array.-------- In natural language processing, a word might be expressed as a vector v {\displaystyle v} via the Word2vec algorithm. Thus v {\displaystyle v} becomes a mode-1 tensor v ↦ A ∈ R N . {\displaystyle v\mapsto {\mathcal {A}}\in \mathbb {R} ^{N}.} The embedding of subject-object-verb semantics requires embedding relationships among three words. Because a word is itself a vector, subject-object-verb semantics could be expressed using mode-3 tensors v a × v b × v c ↦ A ∈ R N × N × N . {\displaystyle v_{a}\times v_{b}\times v_{c}\mapsto {\mathcal {A}}\in \mathbb {R} ^{N\times N\times N}.} In practice the neural network designer is primarily concerned with the specification of embeddings, the connection of tensor layers, and the operations performed on them in a network. Modern machine learning frameworks manage the optimization, tensor factorization and backpropagation automatically. === As unit values === Tensors may be used as the unit values of neural networks which extend the concept of scalar, vector and matrix values to multiple dimensions. The output value of single layer unit y m {\displaystyle y_{m}} is the sum-product of its input units and the connection weights filtered through the activation function f {\displaystyle f} : y m = f ( ∑ n x n u m , n ) , {\displaystyle y_{m}=f\left(\sum _{n}x_{n}u_{m,n}\right),} where y m ∈ R .