Backup

In information technology, a backup, or data backup is a copy of computer data taken and stored elsewhere so that it may be used to restore the original after a data loss event. The verb form, referring to the process of doing so, is "back up", whereas the noun and adjective form is "backup". Backups can be used to recover data after its loss from data deletion or corruption, or to recover data from an earlier time. Backups provide a simple form of IT disaster recovery; however not all backup systems are able to reconstitute a computer system or other complex configuration such as a computer cluster, active directory server, or database server. A backup system contains at least one copy of all data considered worth saving. The data storage requirements can be large. An information repository model may be used to provide structure to this storage. There are different types of data storage devices used for copying backups of data that is already in secondary storage onto archive files. There are also different ways these devices can be arranged to provide geographic dispersion, data security, and portability. Data is selected, extracted, and manipulated for storage. The process can include methods for dealing with live data, including open files, as well as compression, encryption, and de-duplication. Additional techniques apply to enterprise client-server backup. Backup schemes may include dry runs that validate the reliability of the data being backed up. There are limitations and human factors involved in any backup scheme. == Storage == A backup strategy requires an information repository, "a secondary storage space for data" that aggregates backups of data "sources". The repository could be as simple as a list of all backup media (DVDs, etc.) and the dates produced, or could include a computerized index, catalog, or relational database. === 3-2-1 Backup Rule === The backup data needs to be stored, requiring a backup rotation scheme, which is a system of backing up data to computer media that limits the number of backups of different dates retained separately, by appropriate re-use of the data storage media by overwriting of backups no longer needed. The scheme determines how and when each piece of removable storage is used for a backup operation and how long it is retained once it has backup data stored on it. The 3-2-1 rule can aid in the backup process. It states that there should be at least 3 copies of the data, stored on 2 different types of storage media, and one copy should be kept offsite, in a remote location (this can include cloud storage). 2 or more different media should be used to eliminate data loss due to similar reasons (for example, optical discs may tolerate being underwater while LTO tapes may not, and SSDs cannot fail due to head crashes or damaged spindle motors since they do not have any moving parts, unlike hard drives). An offsite copy protects against fire, theft of physical media (such as tapes or discs) and natural disasters like floods and earthquakes. Physically protected hard drives are an alternative to an offsite copy, but they have limitations like only being able to resist fire for a limited period of time, so an offsite copy still remains as the ideal choice. Because there is no perfect storage, many backup experts recommend maintaining a second copy on a local physical device, even if the data is also backed up offsite. === Backup methods === ==== Unstructured ==== An unstructured repository may simply be a stack of tapes, DVD-Rs or external HDDs with minimal information about what was backed up and when. This method is the easiest to implement, but unlikely to achieve a high level of recoverability as it lacks automation. ==== Full only/System imaging ==== A repository using this backup method contains complete source data copies taken at one or more specific points in time. Copying system images, this method is frequently used by computer technicians to record known good configurations. However, imaging is generally more useful as a way of deploying a standard configuration to many systems rather than as a tool for making ongoing backups of diverse systems. ==== Incremental ==== An incremental backup stores data changed since a reference point in time. Duplicate copies of unchanged data are not copied. Typically a full backup of all files is made once or at infrequent intervals, serving as the reference point for an incremental repository. Subsequently, a number of incremental backups are made after successive time periods. Restores begin with the last full backup and then apply the incrementals. Some backup systems can create a synthetic full backup from a series of incrementals, thus providing the equivalent of frequently doing a full backup. When done to modify a single archive file, this speeds restores of recent versions of files. ==== Near-CDP ==== Continuous Data Protection (CDP) refers to a backup that instantly saves a copy of every change made to the data. This allows restoration of data to any point in time and is the most comprehensive and advanced data protection. Near-CDP backup applications—often marketed as "CDP"—automatically take incremental backups at a specific interval, for example every 15 minutes, one hour, or 24 hours. They can therefore only allow restores to an interval boundary. Near-CDP backup applications use journaling and are typically based on periodic "snapshots", read-only copies of the data frozen at a particular point in time. Near-CDP (except for Apple Time Machine) intent-logs every change on the host system, often by saving byte or block-level differences rather than file-level differences. This backup method differs from simple disk mirroring in that it enables a roll-back of the log and thus a restoration of old images of data. Intent-logging allows precautions for the consistency of live data, protecting self-consistent files but requiring applications "be quiesced and made ready for backup." Near-CDP is more practicable for ordinary personal backup applications, as opposed to true CDP, which must be run in conjunction with a virtual machine or equivalent and is therefore generally used in enterprise client-server backups. Software may create copies of individual files such as written documents, multimedia projects, or user preferences, to prevent failed write events caused by power outages, operating system crashes, or exhausted disk space, from causing data loss. A common implementation is an appended ".bak" extension to the file name. ==== Reverse incremental ==== A Reverse incremental backup method stores a recent archive file "mirror" of the source data and a series of differences between the "mirror" in its current state and its previous states. A reverse incremental backup method starts with a non-image full backup. After the full backup is performed, the system periodically synchronizes the full backup with the live copy, while storing the data necessary to reconstruct older versions. This can either be done using hard links—as Apple Time Machine does, or using binary diffs. ==== Differential ==== A differential backup saves only the data that has changed since the last full backup. This means a maximum of two backups from the repository are used to restore the data. However, as time from the last full backup (and thus the accumulated changes in data) increases, so does the time to perform the differential backup. Restoring an entire system requires starting from the most recent full backup and then applying just the last differential backup. A differential backup copies files that have been created or changed since the last full backup, regardless of whether any other differential backups have been made since, whereas an incremental backup copies files that have been created or changed since the most recent backup of any type (full or incremental). Changes in files may be detected through a more recent date/time of last modification file attribute, and/or changes in file size. Other variations of incremental backup include multi-level incrementals and block-level incrementals that compare parts of files instead of just entire files. === Storage media === Regardless of the repository model that is used, the data has to be copied onto an archive file data storage medium. The medium used is also referred to as the type of backup destination. ==== Magnetic tape ==== Magnetic tape was for a long time the most commonly used medium for bulk data storage, backup, archiving, and interchange. It was previously a less expensive option, but this is no longer the case for smaller amounts of data. Tape is a sequential access medium, so the rate of continuously writing or reading data can be very fast. While tape media itself has a low cost per space, tape drives are typically dozens of times as expensive as hard disk drives and optical drives. Tape media are generally rotated on a schedule so at least one set is off-site in case something should happe

Core FTP

Core FTP LE is a freeware secure FTP client for Windows, developed by CoreFTP.com. Features include FTP, SSL/TLS, SFTP via SSH, and HTTP/HTTPS support. Secure FTP clients encrypt account information and data transferred across the internet, protecting data from being seen, or sniffed across networks. Core FTP is a traditional FTP client with local files displayed on the left, remote files on the right. Core FTP Server is a secure FTP server for Windows, developed by CoreFTP.com, starting in 2010. == Licensing == CoreFTP LE is free for personal, educational, non-profit, and business use.

Nature Manifesto

Nature Manifesto is an Immersive sound piece and multimedia installation by Icelandic artist Björk and artist and curator Aleph Molinari, created in collaboration with the French Institute for Research and Coordination in Acoustics/Music (IRCAM). The installation was showcased at the Centre Pompidou in Paris, France from November 20, 2024 to December 9, 2024, as part of the museum's "Biodiversity: Which Culture for Which Future?" forum. It combines natural soundscapes, calls of extinct animals reconstructed through artificial intelligence, and Björk's narration to address damages to biodiversity and the collapse of ecosystems. == Background == Björk's work intricately weaves themes of nature and technology, reflecting her deep engagement with both realms. In 2008, she co-founded the Náttúra campaign to protest the construction of foreign-backed aluminum factories in Iceland, aiming to protect the country's natural landscapes. She released the single "Náttúra" featuring Thom Yorke, with all proceeds supporting this environmental initiative. Her 2011 album Biophilia further exemplifies this synthesis, exploring the relationships between music, nature, and technology through a multimedia project that included interactive apps, custom-made instruments, and educational workshops. Björk's Cornucopia tour (2019-2023) seamlessly integrates themes of nature preservation and environmental activism, and featured a recorded message by Swedish climate activist Greta Thunberg. The tour's fusion of music, technology, and natural imagery reflects Björk's vision of a harmonious coexistence between humanity and nature, advocating for sustainable futures. Björk has previously used artificial intelligence in her works. In 2020, she collaborated with Microsoft to create Kórsafn, a sound installation for the Sister City Hotel lobby in New York City which used an AI-powered model that elaborated choral recordings from her discography through a sensor on the rooftop of the building that would generate music according to data like the weather and the seasons. For her charity single "Oral", featuring Spanish singer Rosalía, she released a music video directed by photographer and visual artist Carlota Guerrero, who used AI-generated deepfake versions of the artists. == Concept == Nature Manifesto is a three-minute and forty-second immersive sound piece. The composition merges Björk's voice, as she articulates a manifesto on biodiversity and the climate crisis, with cries of extinct and endangered animals, harmonizing them with natural soundscapes. The installation was curated by Chloé Siganos and Aleph Molinari, with associate curator Delphine Le Gatt. The primary goal of Nature Manifesto is to foster a deeper understanding of humanity's impact on the natural world. Conceived as a "post-optimistic" manifesto, Aleph Molinari stated that the project's purpose was to "offer a voice to nature". He stated that "the modern concept of nature itself is problematic [...] because it’s a concept born in the Romantic period and, with the rise of the industrial era, became an antithesis to human civilisation and everything urban. Nature came to define what was outside, the savage Other... But nature is everything that we’re part of." The soundscape features recreated calls of extinct and endangered species, developed in collaboration with the French sound research institute IRCAM. Artificial intelligence was employed to simulate the vocalizations of animals that no longer exist in the wild. To save energy and lessen the ecological impact of the use of AI, the research institute developed a "frugal AI" model capable of generating audio in real-time on local servers without a graphics processing unit. The sounds were then produced and edited by Björk in collaboration with Robin Meier Wiratunga and Bergur Þórisson. The installation was located within the Centre Pompidou's escalator, known as the "caterpillar". The installation was further supported by videos created by visual artist Sam Balfus (also known as Balfua) by using artificial intelligence, and edited by Santiago Molinari. == Activism == To sustain and broaden the themes presented in Nature Manifesto, Björk publicly urged French President Emmanuel Macron to prohibit bottom trawling within France's marine protected areas (MPA). She criticized the French government's claim of protecting 30% of its marine territories, highlighting that over 90% of these MPAs exist only on paper, allowing destructive practices like bottom trawling to continue unchecked. She collaborated with non-governmental organizations Sustainable Ocean Alliance, Ungir umhverfissinnar and Bloom, to advocate for genuine ocean conservation. Björk promoted the cause through her social media profiles by sharing petitions. In November 2024, Björk lent her Instagram account to French environmental activists to directly address Macron. The activists used the platform to call for stronger protection of the ocean, urging Macron to impose stricter restrictions on harmful fishing practices, particularly bottom trawling. == Reception == Nature Manifesto received mixed to positive reviews from critics. Some critiques focused on the installation's setting, suggesting that the movement inherent to the escalator space diminished the immersive potential of the soundscape. The choice of using artificial intelligence was also questioned. Björk and Molinari defended this, as both see AI as a tool that can be used creatively and sustainably, with Björk focusing on the importance of human input to give AI a "soul", and Molinari stressing the need for sustainable technological practices in the broader context of digital life. After the exhibition ended, Björk further opinionated: "this is how we will work in the future. [...] if there is no soul in tomorrow's music made by AI it is because [no one] put it there and we have to speak out and guard this as listeners", further stating that there is already "soulless muzak" [sic] on Spotify, "mass manufactured without the attention of creativity".

Competitions and prizes in artificial intelligence

There are a number of competitions and prizes to promote research in artificial intelligence. == General machine intelligence == The David E. Rumelhart Prize is an annual award for making a "significant contemporary contribution to the theoretical foundations of human cognition". The prize is $100,000. The Human-Competitive Award is an annual challenge started in 2004 to reward results "competitive with the work of creative and inventive humans". The prize is $10,000. Entries are required to use evolutionary computing. The Intel AI Global Impact Festival is an international annual competition held by Intel Corporation for school, and college students with prizes upwards of $15,000. It is about artificial intelligence technology. There are two age brackets in this competition, 13-18 Age Group, and 18 and Above Age Group. The IJCAI Award for Research Excellence is a biannual award given at the International Joint Conference on Artificial Intelligence (IJCAI) to researchers in artificial intelligence as a recognition of excellence of their career. The 2011 Federal Virtual World Challenge, advertised by The White House and sponsored by the U.S. Army Research Laboratory's Simulation and Training Technology Center, held a competition offering a total of US$52,000 in cash prize awards for general artificial intelligence applications, including "adaptive learning systems, intelligent conversational bots, adaptive behavior (objects or processes)" and more. The Machine Intelligence Prize is awarded annually by the British Computer Society for progress towards machine intelligence. The Kaggle – "the world's largest community of data scientists compete to solve most valuable problems". == Conversational behaviour == The Loebner prize is an annual competition to determine the best Turing test competitors. The winner is the computer system that, in the judges' opinions, demonstrates the "most human" conversational behaviour, they have an additional prize for a system that in their opinion passes a Turing test. This second prize has not yet been awarded. == Automatic control == === Pilotless aircraft === The International Aerial Robotics Competition is a long-running event begun in 1991 to advance the state of the art in fully autonomous air vehicles. This competition is restricted to university teams (although industry and governmental sponsorship of teams is allowed). Key to this event is the creation of flying robots which must complete complex missions without any human intervention. Successful entries are able to interpret their environment and make real-time decisions based only on a high-level mission directive (e.g., "find a particular target inside a building having certain characteristics which is among a group of buildings 3 kilometers from the aerial robot launch point"). In 2000, a $30,000 prize was awarded during the 3rd Mission (search and rescue), and in 2008, $80,000 in prize money was awarded at the conclusion of the 4th Mission (urban reconnaissance). === Driverless cars === The DARPA Grand Challenge is a series of competitions to promote driverless car technology, aimed at a congressional mandate stating that by 2015 one-third of the operational ground combat vehicles of the US Armed Forces should be unmanned. While the first race had no winner, the second awarded a $2 million prize for the autonomous navigation of a hundred-mile trail, using GPS, computers and a sophisticated array of sensors. In November 2007, DARPA introduced the DARPA Urban Challenge, a sixty-mile urban area race requiring vehicles to navigate through traffic. In November 2010 the US Armed Forces extended the competition with the $1.6 million prize Multi Autonomous Ground-robotic International Challenge to consider cooperation between multiple vehicles in a simulated-combat situation. Roborace will be a global motorsport championship with autonomously driving, electric vehicles. The series will be run as a support series during the Formula E championship for electric vehicles. This will be the first global championship for driverless cars. == Data-mining and prediction == The Netflix Prize was a competition for the best collaborative filtering algorithm that predicts user ratings for films, based on previous ratings. The competition was held by Netflix, an online DVD-rental service. The prize was $1,000,000. The Pittsburgh Brain Activity Interpretation Competition will reward analysis of fMRI data "to predict what individuals perceive and how they act and feel in a novel Virtual Reality world involving searching for and collecting objects, interpreting changing instructions, and avoiding a threatening dog." The prize in 2007 was $22,000. The Face Recognition Grand Challenge (May 2004 to March 2006) aimed to promote and advance face recognition technology. The American Meteorological Society's artificial intelligence competition involves learning a classifier to characterise precipitation based on meteorological analyses of environmental conditions and polarimetric radar data. == Cooperation and coordination == === Robot football === The RoboCup and Federation of International Robot-soccer Association (FIRA) are annual international robot soccer competitions. The International RoboCup Federation challenge is by 2050 "a team of fully autonomous humanoid robot soccer players shall win the soccer game, comply with the official rule of the FIFA, against the winner of the most recent World Cup." == Logic, reasoning and knowledge representation == The Herbrand Award is a prize given by Conference on Automated Deduction (CADE) Inc. to honour persons or groups for important contributions to the field of automated deduction. The prize is $1000. The CADE ATP System Competition (CASC) is a yearly competition of fully automated theorem provers for classical first order logic associated with the Conference on Automated Deduction (CADE) and International Joint Conference on Automated Reasoning (IJCAR). The competition was part of the Alan Turing Centenary Conference in 2012, with total prizes of 9000 GBP given by Google. The SUMO prize is an annual prize for the best open source ontology extension of the Suggested Upper Merged Ontology (SUMO), a formal theory of terms and logical definitions describing the world. The prize is $3000. The Hutter Prize for lossless compression of human knowledge is a cash prize which rewards compression improvements on a specific 100 MB English text file. The prize awards 500 euros for each one percent improvement, up to €50,000. The organizers believe that text compression and AI are equivalent problems and 3 prizes have been given, at around € 2k. The Cyc TPTP Challenge is a competition to develop reasoning methods for the Cyc comprehensive ontology and database of everyday common sense knowledge. The prize is 100 euros for "each winner of two related challenges". The Eternity II challenge was a constraint satisfaction problem very similar to the Tetravex game. The objective is to lay 256 tiles on a 16x16 grid while satisfying a number of constraints. The problem is known to be NP-complete. The prize was US$2,000,000. The competition ended in December 2010. == Games == The World Computer Chess Championship has been held since 1970. The International Computer Games Association continues to hold an annual Computer Olympiad which includes this event plus computer competitions for many other games. The Ing Prize was a substantial money prize attached to the World Computer Go Congress, starting from 1985 and expiring in 2000. It was a graduated set of handicap challenges against young professional players with increasing prizes as the handicap was lowered. At the time it expired in 2000, the unclaimed prize was 400,000 NT dollars for winning a 9-stone handicap match. The AAAI General Game Playing Competition is a competition to develop programs that are effective at general game playing. Given a definition of a game, the program must play it effectively without human intervention. Since the game is not known in advance the competitors cannot especially adapt their programs to a particular scenario. The prize in 2006 and 2007 was $10,000. The General Video Game AI Competition (GVGAI) poses the problem of creating artificial intelligence that can play a wide, and in principle unlimited, range of games. Concretely, it tackles the problem of devising an algorithm that is able to play any game it is given, even if the game is not known a priori. Additionally, the contests poses the challenge of creating level and rule generators for any game is given. This area of study can be seen as an approximation of General Artificial Intelligence, with very little room for game dependent heuristics. The competition runs yearly in different tracks: single player planning, two-player planning, single player learning, level and rule generation, and each track prizes ranging from 200 to 500 US dollars for winners and runner-ups. The 2007 Ultimate Computer Ches

International Conference on Autonomous Agents and Multiagent Systems

The International Conference on Autonomous Agents and Multiagent Systems or AAMAS is the leading scientific conference for research in the areas of artificial intelligence, autonomous agents, and multiagent systems. It is annually organized by a non-profit organization called the International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). == History == The International Conference on Autonomous Agents and Multiagent Systems (AAMAS) is a highly respected joint conference that provides a quality forum for discussing research in intelligent computational agents and their interactions. It is a merger of three major international conferences/workshops, namely the International Conference on Autonomous Agents (AGENTS), International Conference on Multi-Agent Systems (ICMAS), and International Workshop on Agent Theories, Architectures, and Languages (ATAL). ICMAS is itself a merger of three formative workshops, each with an attendance of fewer than 50 researchers. At a meeting during IJCAI-93 held in Chambery, France in August 1993, the leaders of the European Workshops on Modelling Autonomous Agents in a Multi-Agent World, the Asian MAAC Workshops, and the North American Distributed Artificial Intelligence Workshops (Victor Lesser, Michael N. Huhns, Les Gasser, Barbara Grosz, Nicholas Jennings, Michael Wooldridge, Gerhard Weiss, Mario Tokoro, and Toru Ishida) began the planning for a combined conference, which resulted in the first ICMAS in San Francisco, CA, USA in 1995, attended by more than 500 researchers. The AAMAS Conference is under the guidance and management of the International Foundation for Autonomous Agents and Multiagent Systems, which is incorporated as a 501(c)(3) non-profit organization in South Carolina, USA. == Current and previous conferences == 2024: Auckland, New Zealand (May 6-10) 2023: London, United Kingdom (May 29-June 1) 2022: Auckland, New Zealand (May 9–13) 2021: London, United Kingdom (May 3-May 7) 2020: Auckland, New Zealand (May 9–13) 2019: Montreal, Canada (May 13–17) 2018: Stockholm, Sweden (July 10–15) 2017: São Paulo, Brazil 2016: Singapore City, Singapore 2015: Istanbul, Turkey 2014: Paris, France 2013: Saint Paul, USA 2012: Valencia, Spain 2011: Taipei, Taiwan 2010: Toronto, Canada 2009: Budapest, Hungary 2008: Estoril, Portugal 2007: Honolulu, USA 2006: Hakodate, Japan 2005: Utrecht, The Netherlands 2004: New York, USA 2003: Melbourne, Australia 2002: Bologna, Italy == Activities == Besides the main program that consists of a main track, an industry and applications track, and a couple of special area tracks, AAMAS also hosts over 20 workshops (e.g., AOSE, COIN, DALT, ProMAS, to mention a few) and many tutorials. There is also a demonstration session and a doctoral symposium. Finally, each year AAMAS features a bunch of awards, most notably the IFAAMAS Influential Paper Award. It publishes proceedings which are available online.

Semantic compression

In natural language processing, semantic compression is a process of compacting a lexicon used to build a textual document (or a set of documents) by reducing language heterogeneity, while maintaining text semantics. As a result, the same ideas can be represented using a smaller set of words. In most applications, semantic compression is a lossy compression. Increased prolixity does not compensate for the lexical compression and an original document cannot be reconstructed in a reverse process. == By generalization == Semantic compression is basically achieved in two steps, using frequency dictionaries and semantic network: determining cumulated term frequencies to identify target lexicon, replacing less frequent terms with their hypernyms (generalization) from target lexicon. Step 1 requires assembling word frequencies and information on semantic relationships, specifically hyponymy. Moving upwards in word hierarchy, a cumulative concept frequency is calculating by adding a sum of hyponyms' frequencies to frequency of their hypernym: c u m f ( k i ) = f ( k i ) + ∑ j c u m f ( k j ) {\displaystyle cumf(k_{i})=f(k_{i})+\sum _{j}cumf(k_{j})} where k i {\displaystyle k_{i}} is a hypernym of k j {\displaystyle k_{j}} . Then a desired number of words with top cumulated frequencies are chosen to build a target lexicon. In the second step, compression mapping rules are defined for the remaining words in order to handle every occurrence of a less frequent hyponym as its hypernym in output text. Example The below fragment of text has been processed by the semantic compression. Words in bold have been replaced by their hypernyms. They are both nest building social insects, but paper wasps and honey bees organize their colonies in very different ways. In a new study, researchers report that despite their differences, these insects rely on the same network of genes to guide their social behavior.The study appears in the Proceedings of the Royal Society B: Biological Sciences. Honey bees and paper wasps are separated by more than 100 million years of evolution, and there are striking differences in how they divvy up the work of maintaining a colony. The procedure outputs the following text: They are both facility building insect, but insects and honey insects arrange their biological groups in very different structure. In a new study, researchers report that despite their difference of opinions, these insects act the same network of genes to steer their party demeanor. The study appears in the proceeding of the institution bacteria Biological Sciences. Honey insects and insect are separated by more than hundred million years of organic processes, and there are impinging differences of opinions in how they divvy up the work of affirming a biological group. == Implicit semantic compression == A natural tendency to keep natural language expressions concise can be perceived as a form of implicit semantic compression, by omitting unmeaningful words or redundant meaningful words (especially to avoid pleonasms). == Applications and advantages == In the vector space model, compacting a lexicon leads to a reduction of dimensionality, which results in less computational complexity and a positive influence on efficiency. Semantic compression is advantageous in information retrieval tasks, improving their effectiveness (in terms of both precision and recall). This is due to more precise descriptors (reduced effect of language diversity – limited language redundancy, a step towards a controlled dictionary). As in the example above, it is possible to display the output as natural text (re-applying inflexion, adding stop words).

Stable Diffusion

Stable Diffusion is a deep learning, text-to-image model released in 2022 based on diffusion techniques. The generative artificial intelligence technology is the premier product of Stability AI and is considered to be a part of the ongoing AI boom. It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt. Its development involved researchers from the CompVis Group at LMU Munich and Runway with a computational donation from Stability and training data from non-profit organizations. Stable Diffusion is a latent diffusion model, a kind of deep generative artificial neural network. Its code and model weights have been released publicly, and an optimized version can run on most consumer hardware equipped with a modest GPU with as little as 2.4 GB VRAM. This marked a departure from previous proprietary text-to-image models such as DALL-E and Midjourney which were accessible only via cloud services. == Development == Stable Diffusion originated from a project called Latent Diffusion, developed in Germany by researchers at LMU Munich in Munich and Heidelberg University. Four of the original 5 authors (Robin Rombach, Andreas Blattmann, Patrick Esser and Dominik Lorenz) later joined Stability AI and released subsequent versions of Stable Diffusion. The technical license for the model was released by the CompVis group at LMU Munich. Development was led by Patrick Esser of Runway and Robin Rombach of CompVis, who were among the researchers who had earlier invented the latent diffusion model architecture used by Stable Diffusion. Stability AI also credited EleutherAI and LAION (a German nonprofit which assembled the dataset on which Stable Diffusion was trained) as supporters of the project. == Technology == === Architecture === Diffusion models, introduced in 2015, are trained with the objective of removing successive applications of Gaussian noise on training images, which can be thought of as a sequence of denoising autoencoders. The name diffusion is from the thermodynamic diffusion, since they were first developed with inspiration from thermodynamics. Models in Stable Diffusion series before SD 3 all used a variant of diffusion models, called latent diffusion model (LDM), developed in 2021 by the CompVis (Computer Vision & Learning) group at LMU Munich. Stable Diffusion consists of 3 parts: the variational autoencoder (VAE), U-Net, and an optional text encoder. The VAE encoder compresses the image from pixel space to a smaller dimensional latent space, capturing a more fundamental semantic meaning of the image. Gaussian noise is iteratively applied to the compressed latent representation during forward diffusion. The U-Net block, composed of a ResNet backbone, denoises the output from forward diffusion backwards to obtain a latent representation. Finally, the VAE decoder generates the final image by converting the representation back into pixel space. The denoising step can be flexibly conditioned on a string of text, an image, or another modality. The encoded conditioning data is exposed to denoising U-Nets via a cross-attention mechanism. For conditioning on text, the fixed, pretrained CLIP ViT-L/14 text encoder is used to transform text prompts to an embedding space. Researchers point to increased computational efficiency for training and generation as an advantage of LDMs. With 860 million parameters in the U-Net and 123 million in the text encoder, Stable Diffusion is considered relatively lightweight by 2022 standards, and unlike other diffusion models, it can run on consumer GPUs, and even CPU-only if using the OpenVINO version of Stable Diffusion. ==== SD XL ==== The XL version uses the same LDM architecture as previous versions, except larger: larger UNet backbone, larger cross-attention context, two text encoders instead of one, and trained on multiple aspect ratios (not just the square aspect ratio like previous versions). The SD XL Refiner, released at the same time, has the same architecture as SD XL, but it was trained for adding fine details to preexisting images via text-conditional img2img. ==== SD 3.0 ==== The 3.0 version completely changes the backbone. Not a UNet, but a Rectified Flow Transformer, which implements the rectified flow method with a Transformer. The Transformer architecture used for SD 3.0 has three "tracks", for original text encoding, transformed text encoding, and image encoding (in latent space). The transformed text encoding and image encoding are mixed during each transformer block. The architecture is named "multimodal diffusion transformer (MMDiT), where the "multimodal" means that it mixes text and image encodings inside its operations. This differs from previous versions of DiT, where the text encoding affects the image encoding, but not vice versa. === Training data === Stable Diffusion was trained on pairs of images and captions taken from LAION-5B, a publicly available dataset derived from Common Crawl data scraped from the web, where 5 billion image-text pairs were classified based on language and filtered into separate datasets by resolution, a predicted likelihood of containing a watermark, and predicted "aesthetic" score (e.g. subjective visual quality). The dataset was created by LAION, a German non-profit which receives funding from Stability AI. The Stable Diffusion model was trained on three subsets of LAION-5B: laion2B-en, laion-high-resolution, and laion-aesthetics v2 5+. A third-party analysis of the model's training data identified that out of a smaller subset of 12 million images taken from the original wider dataset used, approximately 47% of the sample size of images came from 100 different domains, with Pinterest taking up 8.5% of the subset, followed by websites such as WordPress, Blogspot, Flickr, DeviantArt and Wikimedia Commons. An investigation by Bayerischer Rundfunk showed that LAION's datasets, hosted on Hugging Face, contain large amounts of private and sensitive data. === Training procedures === The model was initially trained on the laion2B-en and laion-high-resolution subsets, with the last few rounds of training done on LAION-Aesthetics v2 5+, a subset of 600 million captioned images which the LAION-Aesthetics Predictor V2 predicted that humans would, on average, give a score of at least 5 out of 10 when asked to rate how much they liked them. The LAION-Aesthetics v2 5+ subset also excluded low-resolution images and images which LAION-5B-WatermarkDetection identified as carrying a watermark with greater than 80% probability. Final rounds of training additionally dropped 10% of text conditioning to improve Classifier-Free Diffusion Guidance. The model was trained using 256 Nvidia A100 GPUs on Amazon Web Services for a total of 150,000 GPU-hours, at a cost of $600,000. === Limitations === Stable Diffusion has issues with degradation and inaccuracies in certain scenarios. Initial releases of the model were trained on a dataset that consists of 512×512 resolution images, meaning that the quality of generated images noticeably degrades when user specifications deviate from its "expected" 512×512 resolution; the version 2.0 update of the Stable Diffusion model later introduced the ability to natively generate images at 768×768 resolution. Another challenge is in generating human limbs due to poor data quality of limbs in the LAION database. The model is insufficiently trained to replicate human limbs and faces due to the lack of representative features in the database, and prompting the model to generate images of such type can confound the model. In addition to human limbs, Stable Diffusion is unable to generate legible ambigrams and some other forms of text and typography. Stable Diffusion XL (SDXL) version 1.0, released in July 2023, introduced native 1024x1024 resolution and improved generation for limbs and text. Accessibility for individual developers can also be a problem. In order to customize the model for new use cases that are not included in the dataset, such as generating anime characters ("waifu diffusion"), new data and further training are required. Fine-tuned adaptations of Stable Diffusion created through additional retraining have been used for a variety of different use-cases, from medical imaging to algorithmically generated music. However, this fine-tuning process is sensitive to the quality of new data; low resolution images or different resolutions from the original data can not only fail to learn the new task but degrade the overall performance of the model. Even when the model is additionally trained on high quality images, it is difficult for individuals to run models in consumer electronics. For example, the training process for waifu-diffusion requires a minimum 30 GB of VRAM, which exceeds the usual resource provided in such consumer GPUs as Nvidia's GeForce 30 series, w