[Paper]MOE : Mixture of Experts Explained
Towards Understanding Mixture of Experts in Deep Learning
Huggingface : Mixture of Experts Explained
간결, 명확, 중복 없게, 충돌 없게, 영어로, Naming에도 정성을
언어모델 학습 시켜보기. 그리고 특정 인풋이 들어갈때 활성화되는 가중치 시각화해보기. XAI. 그다음 직접 만든 모델로 LoRA 등 어댑터 적용해보기. 엔드투엔드로 한번 만들어보기.
작성시 목차에 의존하지 말고 내가 직접 생각하면서 정리하기! 만약 목차대로 작성한다면 그냥 “깜지”랑 다를게 없지 않은가?
해당 포스트는 공부를 위해 개인적으로 정리한 내용으로 해당 도서에는 다양한 예시를 통해 좀 더 직관적인 이해가 가능합니다.
ⅠⅡⅢⅣⅤⅥⅦⅧⅨⅩⅪⅫ ⅰⅱⅲⅳⅴⅵⅶⅷⅸⅹⅺⅻ ⒈⒉⒊⒋⒌⒍⒎⒏⒐⒑⒒⒓ ⑴⑵⑶⑷⑸⑹⑺⑻⑼⑽⑾⑿ Α α, Β β, Γ γ, Δ δ, Ε ε, Ζ ζ, Η η, Θ θ, Ι ι, Κ κ, Λ λ, Μ μ, Ν ν, Ξ ξ, Ο ο, Π π, Ρ ρ, Σ σ/ς, Τ τ, Υ υ, Φ φ, Χ χ, Ψ ψ, Ω ω.⋅
Statistics
- 대과목 Course
- 주제 Open Source
- Markdown : marp, Latex, Mermaid Paper
ML
- Graph python
- Guide : PEP
- library : argparse Challenge
- Certificate
- Project DevOps
- OS : linux
- Version Management : git
- Container : Docker, Kube ETC.
- BLOG
- Career
- Book Review
ML 트리에 맞춰서 블로그 포스트 작성 Machine Learning ├── Supervised │ ├── Dimensionality Reduction │ │ └── Linear Discriminant Analysis (LDA) │ ├── Regression │ │ ├── Linear Regression │ │ ├── Multivariate Adaptive Regression Splines (MARS) │ │ ├── K-Nearest Neighbors Regression (KNN) │ │ ├── Random Forest Regression │ │ ├── Decision Tree Regression (CART) │ │ ├── Support Vector Regression (SVR) │ │ └── Locally Weighted Scatterplot Smoothing (LOWESS) │ └── Classification │ ├── Decision Tree Classification (CART) │ ├── Random Forest Classification │ ├── Adaptive Boosting (AdaBoost) │ ├── Gradient Boosted Trees │ ├── Extreme Gradient Boosting (XGBoost) │ ├── K-Nearest Neighbors Classification (KNN) │ ├── Logistic Regression │ ├── Naive Bayes │ └── Support Vector Machines (SVM) ├── Neural Networks │ ├── Generative Adversarial Network │ │ ├── Wasserstein GAN (WGAN) │ │ ├── Cycle GAN │ │ ├── Deep Convolutional GAN (DCGAN) │ │ ├── Conditional GAN (cGAN) │ │ └── Generative Adversarial Network (GAN) │ ├── Autoencoders │ │ ├── Sparse Autoencoder (SAE) │ │ ├── Denoising Autoencoder (DAE) │ │ ├── Variational Autoencoder (VAE) │ │ └── Undercomplete Autoencoder (AE) │ ├── Recurrent Neural Networks │ │ ├── Recurrent Neural Network (RNN) │ │ ├── Long Short Term Memory (LSTM) │ │ └── Gated Recurrent Unit (GRU) │ ├── Feedforward Neural Networks │ │ ├── Deep Feed Forward (DFF) │ │ └── Feed Forward (FF) │ └── Convolutional Neural Networks │ ├── Transposed Convolutional Network │ └── Deep Convolutional Network (DCN) ├── Unsupervised │ ├── Dimensionality Reduction │ │ ├── LLE Embedding (Locally Linear Embedding) │ │ ├── t-SNE (t-Distributed Stochastic Neighbor Embedding) │ │ ├── UMAP (Uniform Manifold Approximation and Projection) │ │ ├── PCA (Principal Component Analysis) │ │ ├── MDS (Multidimensional Scaling) │ └── Clustering │ ├── K-Means │ ├── Gaussian Mixture Models (GMM) │ ├── Hierarchical Agglomerative Clustering (HAC) │ └── DBSCAN (Density-Based Spatial Clustering of Applications with Noise) ├── Reinforcement │ ├── Value Based Method │ │ ├── SARSA (State-Action-Reward-State-Action) │ │ ├── Q-Learning │ │ └── Deep Q Neural Network (DQN) │ └── Policy Based Method │ ├── Policy Gradient (REINFORCE) │ └── Proximal Policy Optimization (PPO) ├── Semi-Supervised │ ├── Label Spreading │ ├── Self Training Classifier │ └── Label Propagation └── Other └── Probabilistic Graphical Models └── Bayesian Belief Networks (BBN)
출처 : https://towardsdatascience.com/denoising-autoencoders-dae-how-to-use-neural-networks-to-clean-up-your-data-cd9c19bc6915
. REFERENCES
- temp
- temp
- temp
- temp