Preweek Gen Ai

This preweek bootcamp notes is a summary of the week's activities and the tasks that were completed the bootcam is organized by Andrew Brown.

Predictive ML VS GEN AI

GenAI Architecture

Rola show simple Architecture for Gen Ai to a more complex archticuture and how the compontens

Different Levels of Openness in Large Language Models (LLMs)

Large language models (LLMs) vary in openness, ranging from fully open-source to partially open models. Below are the main categories:

Fully Open-Source LLMs

These models share all aspects, including code, weights, and training data under permissive licenses.

Examples:

• BLOOM: A 176-billion parameter multilingual model supporting 46 natural languages and 13 programming languages.
• GPT-NeoX and GPT-J: Developed by EleutherAI, with 20 billion and 6 billion parameters respectively.

Open-Weight LLMs

These models disclose the model weights but may not share the training data or full architecture.

Examples:

• LLaMA 3.1: Meta AI's model with variants ranging from 8 billion to 405 billion parameters.
• Falcon 180B: Developed by Technology Innovation Institute (TII), recognized for its performance in the open-source community.

Open-Architecture LLMs

These models share the architecture but may keep the weights and training data proprietary.

Example:

• OPT (Open Pretrained Transformer): A suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters.

Partially Open LLMs

These models may have some components open while others remain proprietary.

Example:

• XGen-7B: Developed by Salesforce, with most variants available for commercial and research purposes, except for specific instructional data-trained versions.

Level of Archticting

• Conceptual — a high level diagram that is used to communicate to key stakeholders the business solution we are implementing

• Logical — a mid level diagram that describes the key technical components but not requiring detailed parameters so we can quickly rearchitect and communicate to our technical team the current workload

  .Physical — a low level diagram that details all possible parameters and connections used by engineers/developers to accurately implement a solution (e.g. ARNs for resources, IP addresses, etc)**

https://medium.com/@nolomokgosi/conceptual-logical-and-physical-design-c24100846931