Natural Language UnderstandingProgramming and Environment


原 PPT 独立图片资源
Natural Language
Understanding

Programming and Environment
Mingming Sun
BIMSA

Outline


原 PPT 独立图片资源
Outline
Why Python
Programming Techniques
Soft Development Skills
Experimental Environment
Suggested Readings

0.2 Python Programming 0.2.1 Why Python



原 PPT 独立图片资源

0.2 Python Programming 0.2.1 Why Python

原 PPT 独立图片资源

Why Python?


原 PPT 独立图片资源
Why Python?

Most Popular Programming Languages Survey
Programming languages popularity in 2024: insights from the TIOBE index
https://www.researchgate.net/publication/387584210_Popularity_of_programming_languages

Efficient for Human


原 PPT 独立图片资源
Efficient for Human
Easy to learn, read, write, and run;
Interactive environment:
Quick experiment;
Easy to debug;
Great IDE:
Jupyter lab and notebook
Easy to merge into current system:
IPython, JPython

Compiler VS Interpreter VS Assembler 1
Compiler VS Interpreter VS Assembler 2

Efficient for Machine


原 PPT 独立图片资源
Efficient for Machine
Why python can be used in high performance area, such AI?
What is the Pareto Principle? | IxDF
原 PPT 独立图片资源
http://blog.behnel.de/posts/cython-pybind11-cffi-which-tool-to-choose.html

Efficient for Machine: Cython


原 PPT 独立图片资源
Efficient for Machine: Cython
Montecarlo
https://inlocrobotics.com/en/optimizing-python-with-cython/

Efficient for Machine: Pybind11


原 PPT 独立图片资源
Efficient for Machine: Pybind11
原 PPT 独立图片资源

Great Ecosystem and Community


原 PPT 独立图片资源
Great Ecosystem and Community
Open-source & Accessible: Free to use, with a vast, growing ecosystem.
Rich Libraries & Frameworks:
Data Science: pandas, NumPy
Machine Learning: Scikit-learn, PyTorch
Web Development: Django, Flask
Thriving Community:
Extensive documentation, forums, and tutorials.
Active contributors and collaborative development

0.2 Python Programming 0.2.2 Programming Techniques



原 PPT 独立图片资源

0.2 Python Programming 0.2.2 Programming Techniques

原 PPT 独立图片资源

Data Processing and Analysis (I)


原 PPT 独立图片资源
Data Processing and Analysis (I)
Regex
A powerful tool for pattern matching and text processing.
Helps with tasks like searching, replacing, or validating strings.

Json (JavaScript Object Notation)
A lightweight data format.
Commonly used for serialize data with multiple attributes, for example, sample with feature and labels.

原 PPT 独立图片资源
原 PPT 独立图片资源

Data Processing and Analysis (II)


原 PPT 独立图片资源
Data Processing and Analysis (II)
Pandas: Table Data Processing

Example of Pandas DataFrame
Force-directed network visualization of the Quaker data, created in Gephi
NetworkX: Graph Data Processing
https://programminghistorian.org/en/lessons/exploring-and-analyzing-network-data-with-python
https://codingsight.com/an-overview-of-dataframes-in-python/

Numerical Computing and Machine Learning


原 PPT 独立图片资源
Numerical Computing and Machine Learning
Numpy: Powerful Numerical Computations

原 PPT 独立图片资源
https://www.techprofree.com/most-important-features-of-scikit-learn/
numpy
Scikit-Learn: Machine Learning Made Simple
https://decodingdatascience.com/numpy-the-foundation-of-data-science-and-ai/

Deep Learning


原 PPT 独立图片资源
Deep Learning

Pytorch: Deep Learning Framework
Huggingface: Platform for Deep Learning
PyTorch modules
A table describing hugging face datasets, models, and spaces
https://www.scalablepath.com/machine-learning/hugging-face
https://www.altexsoft.com/blog/pytorch-library/

Visualization


原 PPT 独立图片资源
Visualization
Matplotlib:
Visualization library for data science, machine learning, and scientific research for data exploration and presentation.

原 PPT 独立图片资源
https://medium.com/@aakash_7/data-visualization-with-matplotlib-d7967ee42fce

GUI


原 PPT 独立图片资源
GUI
Streamlit
An open-source Python library that makes it easy to create interactive web applications for data science and machine learning projects.
Allows you to turn Python scripts into web apps with minimal code.
Ideal for quickly sharing results, visualizations, and models in a user-friendly interface.

图形用户界面, 网站

AI 生成的内容可能不正确。
https://blog.streamlit.io/introducing-streamlit-cloud/

0.2 Python Programming 0.2.3 Software Development Skills



原 PPT 独立图片资源

0.2 Python Programming 0.2.3 Software Development Skills

原 PPT 独立图片资源

Virtual Environment and Package Management


原 PPT 独立图片资源
Virtual Environment and Package Management
A virtual environment is an isolated environment that:
Manage dependencies for different projects separately.
Prevents conflicts between different projects.
Enables reproducibility by isolating project-specific libraries.
Package Management:
It allows you to install, upgrade, and remove packages from your environment.
Recommendation:
Micromamba + Pip

virtual environments
https://www.whiteboxml.com/blog/the-definitive-guide-to-python-virtual-environments-with-conda

Source Code Version Control


原 PPT 独立图片资源
Source Code Version Control
Version control is a system:
Track changes: Easily see what changes have been made to the code, who made them, and why.
Collaboration: Multiple developers can work on the same project without overwriting each other’s work.
History: Revert back to previous versions of the code if needed.
Recommendation: Github

Frame 1 (1)
https://www.legitsecurity.com/blog/github-privilege-escalation-vulnerability

0.2 Python Programming 0.2.4 Experimental Environment



原 PPT 独立图片资源

0.2 Python Programming 0.2.4 Experimental Environment

原 PPT 独立图片资源

Jupyter Lab and Notebook


原 PPT 独立图片资源
Jupyter Lab and Notebook
Lab:
Web-based interactive development environment 
Notebook
An interactive web-based environment for running code, visualizing data, and documenting results.
Ideal for data analysis, machine learning, and educational purposes.

原 PPT 独立图片资源
https://docs.jupyter.org/en/latest/

Experimental Platform


原 PPT 独立图片资源
Experimental Platform
Provide unlimited CPU resource and 100 hours GPU;
With jupyter notebook support;
Student should finish programming assignment by:
Writing document and code in the notebook,
Test the code in the provided computer,
Visualize the result;
Submit the final notebook.

0.2 Python Programming 0.2.5 Suggested Readings



原 PPT 独立图片资源

0.2 Python Programming 0.2.5 Suggested Readings

原 PPT 独立图片资源

Books


原 PPT 独立图片资源
Books
Automated the Boring Stuff with Python, by Al Sweigart, https://automatetheboringstuff.com/
Python for Data Analysis, by Wes McKinney https://wesmckinney.com/book/

Any Questions?



原 PPT 独立图片资源
Any Questions?
原 PPT 独立图片资源