Deep Learning

Sign in

Deep Learning

deep learning in machine learning category

Finetuning llama

For Llama2, refer to Fine-Tuning Llama 2 Using LoRA and QLoRA: A Comprehensive Guide. For training phase * Set up QLoRA and BitsAndBytes configuration * Load base model with quantization setting * Tokenizer and perfconfiguraiton * Start SFTTrainer * Save weight to certain folder after training For evaluation phase * Load base model using AutoModelForCausalLM * Merge

GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Vision, MoE

GPT4, algorithm, ML architect

Transformers — A Visual & Intuitive Guide

Reference: Transformers Explained Visually (Part 1): Overview of Functionality Transformers Explained Visually (Part 2): How it works, step-by-step Transformers Explained Visually (Part 3): Multi-head Attention, deep dive Transformers Explained Visually — Not Just How, but Why They Work So Well Foundations of NLP Explained — Bleu Score and WER Metrics Foundations of

Nvidia-smi fail to communicate with driver

After we might periodically update our linux kernel, sometimes we might confront with "Nvidia-smi failure": even if your driver is working correctly, error message will pop up, when running $nvidia-smi, like following: dluser1@cnpvgl903653:~$ nvidia-smi NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver.