LoRA模型
为了降低大模型微调的成本，微软的研究人员开发了低秩自适应（LoRA）技术。LoRA 的精妙之处在于，它相当于在原有大模型的基础上增加了一个可拆卸的插件，模型主体保持不变。LoRA 随插随用，轻巧方便。。
==== |
====== |
========= |
======= |
===== |
==== |
==== |
==== |
======= |
======= |
============ |
TensorFlow:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
An open-source software library for Machine Intelligence
Versions:
TensorFlow/2.2.2-foss-2019b-Python-3.7.4
TensorFlow/2.8.4-foss-2021b-CUDA-11.4.1
TensorFlow/2.11.0-foss-2022a-CUDA-11.7.0
TensorFlow/2.11.0-foss-2022a
TensorFlow/2.13.0-foss-2022b
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "TensorFlow" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider TensorFlow/2.13.0-foss-2022b
Miniconda3:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
Miniconda is a free minimal installer for conda. It is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on, and a small number of other useful packages.
Versions:
Miniconda3/4.7.10
Miniconda3/4.9.2
Miniconda3/4.10.3
Miniconda3/4.12.0
Miniconda3/22.11.1-1
Miniconda3/23.5.2-0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "Miniconda3" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider Miniconda3/23.5.2-0
PyTorch:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
Tensors and Dynamic neural networks in Python with strong GPU acceleration. PyTorch is a deep learning framework that puts Python first.
Versions:
PyTorch/1.12.0-foss-2022a-CUDA-11.7.0
PyTorch/1.12.1-foss-2021b-CUDA-11.7.0
PyTorch/1.12.1-foss-2022a-CUDA-11.7.0
PyTorch/2.0.1-foss-2022a
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "PyTorch" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider PyTorch/2.0.1-foss-2022a
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
PyTorch-Lightning: PyTorch-Lightning/1.8.4-foss-2022a-CUDA-11.7.0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
PyTorch Lightning is the lightweight PyTorch wrapper for ML researchers.
This module can be loaded directly: module load PyTorch-Lightning/1.8.4-foss-2022a-CUDA-11.7.0
Help:
Description
===========
PyTorch Lightning is the lightweight PyTorch wrapper for ML researchers.
More information
================
- Homepage: https://pytorchlightning.ai
Included extensions
===================
aiobotocore-2.3.4, aioitertools-0.11.0, anyio-3.6.1, arrow-1.2.3,
blessed-1.19.1, botocore-1.24.21, commonmark-0.9.1, croniter-1.3.8,
fastapi-0.79.1, fsspec-2022.7.1, inquirer-2.10.1, itsdangerous-2.1.2,
jmespath-1.0.1, lightning-utilities-0.4.2, lightning_cloud-0.5.12,
pydantic-1.10.2, PyJWT-2.6.0, python-editor-1.0.4, pytorch-lightning-1.8.4,
readchar-4.0.3, rich-12.5.1, s3fs-2022.7.1, sniffio-1.3.0, starlette-0.19.1,
starsessions-1.3.0, torchmetrics-0.9.3, traitlets-5.2.0, websocket-
client-1.4.2, wrapt-1.14.1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Biopython:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
Biopython is a set of freely available tools for biological computation written in Python by an international team of developers. It is a distributed collaborative effort to develop Python libraries and applications which address the needs of current and future work in bioinformatics.
Versions:
Biopython/1.75-foss-2019b-Python-3.7.4
Biopython/1.76-foss-2022a-Python-2.7.18
Biopython/1.78-foss-2020b
Biopython/1.79-foss-2021b
Biopython/1.79-foss-2022a
Biopython/1.81-foss-2022b
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "Biopython" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
Python:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
Python is a programming language that lets you work more quickly and integrate your systems more effectively.
Versions:
Python/2.7.16-GCCcore-8.3.0
Python/2.7.18-GCCcore-9.3.0
Python/2.7.18-GCCcore-10.2.0
Python/2.7.18-GCCcore-10.3.0-bare
Python/2.7.18-GCCcore-11.2.0-bare
Python/2.7.18-GCCcore-11.2.0
Python/2.7.18-GCCcore-11.3.0-bare
Python/2.7.18-GCCcore-11.3.0
Python/2.7.18-GCCcore-12.3.0
Python/3.7.4-GCCcore-8.3.0
Python/3.8.6-GCCcore-10.2.0
Python/3.9.5-GCCcore-10.3.0-bare
Python/3.9.5-GCCcore-10.3.0
Python/3.9.6-GCCcore-11.2.0-bare
Python/3.9.6-GCCcore-11.2.0
Python/3.10.4-GCCcore-11.3.0-bare
Python/3.10.4-GCCcore-11.3.0
Python/3.10.8-GCCcore-12.2.0-bare
Python/3.10.8-GCCcore-12.2.0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "Python" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider Python/3.10.8-GCCcore-12.2.0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pandas: pandas/1.0.5-foss-2022a-Python-3.10.4
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.
Other possible modules matches:
geopandas, pandasql
This module can be loaded directly: module load pandas/1.0.5-foss-2022a-Python-3.10.4
Help:
Description
===========
pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures
and data analysis tools for the Python programming language.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
numpy: numpy/1.9.2-intel-2021b-Python-2.7.18
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
NumPy is the fundamental package for scientific computing with Python. It contains among other things: a powerful N-dimensional array object, sophisticated (broadcasting) functions, tools for integrating C/C++ and Fortran code, useful linear algebra, Fourier transform, and random number capabilities. Besides its obvious
scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases.
This module can be loaded directly: module load numpy/1.9.2-intel-2021b-Python-2.7.18
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
scikit-learn:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Description:
Scikit-learn integrates machine learning algorithms in the tightly-knit scientific Python world, building upon numpy, scipy, and matplotlib. As a machine-learning module, it provides versatile tools for data mining and analysis in any field of science and engineering. It strives to be simple and efficient, accessible to
everybody, and reusable in various contexts.
Versions:
scikit-learn/1.0.1-foss-2021b
scikit-learn/1.1.2-foss-2022a
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "scikit-learn" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider scikit-learn/1.1.2-foss-2022a
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------