LOG.OS
1.0.0
main1.0.0
1.0.0
main1.0.0
  • HOME
  • LOGOS
    • LOGOS
      • 2021
        • 4
        • 5
  • THEORY
    • Math Basics
      • Chapter 2 Groups, Rings & Fields
        • 2.1 Groups, Subgroups, Cosets
    • Information Theory
      • Chapter 2 Entropy, Relative Entropy & Mutual Information
    • Formal Languages
      • Regular Expressions
    • Language Processing
      • Chapter 2 RE, Text Norm, Edit dist.
        • 2.2 Words
        • 2.3 Corpora
        • 2.4 Text Normalization
        • 2.5 Minimum Edit Distance
      • Chapter 3 N-gram Language Models
        • 3.1 N-Grams
        • 3.2 Evaluating Language Models
        • 3.3 Generalization and Zeros
        • 3.4 Smoothing
        • 3.5 Kneser-Ney Smoothing
      • Chapter 4 Naive Bayes and Sentiment Classification
        • 4.1 Naive Bayes Classifiers
        • 4.6 Naive Bayes as a Language Model
        • 4.7 Evaluation: Precision, Recall, F-measure
      • Chapter 5 Logistic Regression
        • 5.3 The Cross-entropy Loss Function
      • Chapter 6 Vector Semantics and Embeddings
        • 6.1 Lexical Semantics
      • Miscellaneous
    • Algorithms
      • LeetCode Notebook
      • Miscellaneous
    • HCI Toolbox
      • Miscellaneous
  • Programming Languages
    • Python
      • File System & I/O
        • Serialization
      • String
      • Containers
        • Emulating Containers
      • Function
      • Class & Object
      • Type & Reflection
      • Module & Package
      • Python Doc
      • Iterator & Generator
      • Iterables
      • Numerical
      • Miscellaneous
    • C
      • String
      • Miscellaneous
    • C#
      • I/O
      • Miscellaneous
    • C51
      • Basics
      • I/O
      • Start Up Files
      • Listing File / LST
      • Compiler Options
      • Assembler Options
      • MAP file / M51
      • Linker Options
      • Miscellaneous
    • C++
      • DLL
      • Miscellaneous
    • Java
      • Introduction
      • Data Types
      • Control Flow
      • String
      • I&O
      • Classes & Data Structures
      • Miscellaneous
    • JavaScript
      • Control Flow
      • String
      • Array
      • Objects
      • Functions
      • Class
      • ArrayBuffer & TypedArray
      • Iterator & Generator
      • import & export
      • Promise
      • Miscellaneous
    • Bash
      • 3.2 Shell Commands
        • 3.2.5 Compound Commands
      • 6.2 Bash Startup Files
      • Miscellaneous
    • CMD & PowerShell
      • Miscellaneous
    • Rust
      • Basics
      • IO & String
      • OOP
      • Miscellaneous
  • TOOLS - META
    • Bash Tools
      • find
      • tar
      • tmux
      • scp
      • rsync
      • Miscellaneous
    • pip
      • Miscellaneous
    • Python Tools
      • Virtual Environment
      • Dependency Parsing
    • Vim
      • Miscellaneous
    • Git
    • CMake
    • Node.js & npm
    • adb
      • Basics
    • SSH
      • Introduction
      • Login to Remote with SSH Keys
      • Port Forwarding
    • Jupyter Notebook
  • TOOLS - GRAPHICAL
    • X11
    • VNCServer
    • scrcpy
    • OpenCV
      • Installation
      • Hello World
      • Mat
    • OpenNI 2
      • Installation
      • Untitled
    • OpenGL
      • Hello OpenGL
      • Pipeline Basics
    • DirectX
    • NVIDIA HairWorks
      • Interface Mapping in Unity Wrapper
    • Azure Kinect DK
      • Introduction
      • First Application
      • Initialization
      • Data Retrieval
        • Image Format Note
      • Image Manipulation
        • Example Practice
  • TOOLS - NUMERICAL
    • MATLAB
      • Basics
      • Linear Algebra
      • Plotting
      • Numerical Analysis
      • Control Theory
      • Statistics
    • Anaconda
      • Installation
      • Virtual Environment
      • Miscellaneous
    • NumPy
      • Array Basics
      • Miscellaneous
    • TensorFlow
      • Source Code
        • Basics
        • Framework
      • Miscellaneous
    • PyTorch
      • Introduction
      • Tensor
      • Dataset & DataLoader
      • Model Building
        • torch.nn
      • Training
      • torchtext
      • Miscellaneous
    • Auxiliary
      • nltk
      • transformers
        • tokenizer
  • TOOLS - VIRTUALIZED
    • LLVM
    • Docker
      • Installation
      • Basics
    • VirtualBox
  • TOOLS - NETWORKING
    • Flask
  • IDE
    • Visual Studio 2017
      • Miscellaneous
    • Unity
      • Lifecycle
      • Geometry
      • Assets and Scenes
      • Layers
      • Camera
      • User Interaction
      • Advanced Graphics
      • Environment & Platform
      • Serialization
      • Networking
      • Concurrency
      • Unity on Android
      • Miscellaneous
    • Keil μvision4
    • Simplicity Studio
    • WeChat Mini Program DK
      • Project Structure
      • WXML
      • Storage
      • Miscellaneous
  • ARCHITECTURE
    • C++ Optimization
    • USB Protocol
      • Miscellaneous
    • BLE Protocol
      • Introduction
    • AVR
    • Arduino
      • Miscellaneous
    • C8051Fxxx
      • Miscellaneous
    • Android
      • Bluetooth
    • WSL
    • Hololens 2
      • Build & Debug
      • Face Tracking
      • Locatable Camera
    • CI/CD Server from start
      • Initialization
Powered by GitBook
On this page

Was this helpful?

  1. TOOLS - NUMERICAL
  2. Auxiliary

transformers

🤗 transformers.

PreviousnltkNexttokenizer

Last updated 3 years ago

Was this helpful?