Changelog
v1.6.0
2026-11-06
Added
Add ModelPackage to create models from MXQ entries in an MXQ package (.mxqpkg) without unpacking it
Add Model::create(mxq_pkg_path, entry_name[, config], sc) and Model::getPackageEntryName
Add getModelSummary(mxq_pkg_path, entry_name) to summarize an MXQ entry in an MXQ package
Embed file version info in Windows qbruntime.dll
Support models with InstructionVariant
Fixed
Hidden/Cell Upload fixed in multi-threaded situation
Fix potential hanging in inference failure case
Run a single-bundle BatchLLM model (e.g. the post-slice expert and lm_head partitions of a batch MoE package) with its BatchParams; it failed with Model_BrokenMXQ because infer took the non-batch path for any one-bundle model
Changed
Reduce NPU memory usage by sharing duplicated weights and LLM relocation blocks
Removed
Remove legacy script/release/
v1.5.0
2026-09-18
Added
Add Model::resetFixedTailCacheMemory to reset only CacheType::FixedTailCache
Add Model::dump/loadCacheMemoryByName and Model::dump/loadCacheMemoryByLayerHash to dump or load a single cache
NPUData is owned by the Accelerator, not the Model; it stays valid after the model is destroyed
Fixed
Fix quantization accuracy loss in float to int8 conversion
Fix ModelConfig::setActivationSlots(0) / (1) being ignored in CoreMode::Auto
Changed
Unify the type-specific infer/inferAsync/repositionInputs overloads into templates
v1.4.0
2026-08-21
Added
Validate NPU-only model I/O order against the NPU op layer
Support multiple REGULUS-USB devices on Windows
CMakeLists.txt : Add MSVC /MT flag for static build
QBRUTNIME_CLANG_TIDY compile option added
Support beta-version of NPUData class
CoreMode::Auto now detects the CoreMode of each bundle independently, so bundles of
different CoreModes can run on the same model
Fixed
Remove deprecated latencies related APIs
Fix static initialization order crash in Windows
Validate SequenceLength without input axes at Model::create
Export statusCodeToString from the shared library
Fix max sequence length
Changed
Migrate HW selection to runtime
Model class no longer creates a temporary directory
regulus-rb MXQ no longer runs on a regulus-rb-usb device, and vice versa
Removed
Remove windows release resources
v1.3.2
2026-07-10
Fixed
Fix wrong I/O base address mapping in initPinnedInferBundlePrepManager by filtering out DDR_IMEM entries from the relocation table
v1.3.1
2026-07-09
Added
Support avx512 for scale and transpose operations
Add QBRUNTIME_NPU_TIMEOUT_MS environment variable
Support cmake Config/Target for find_package()
Fixed
Change timer module from system_clock to steady_clock to prevent timeout malfunction
Fix segfault when running inferSpeedrun of model using variable-length input
Fix wrong initRmem skip for multi-shape model
Fix failure of releasing activation index
Fix bugs of RNN/LSTM models on Regulus SoC
GCC8 build failure resolved
Removed
Remove legacy Aries(v1) driver/product and NOC interleaving support
Support Asymmetric Quantization for Int8 NPU DataType
Remove QBRUNTIME_USE_SSE2_FOR_SIMD option from root CMakeLists.txt
v1.3.0
2026-06-12
Added
Support custom 16bit integer type, BInt16
Support PinnedMemory for Regulus SoC
Support Initstart - Rmem attachment & Rmem broadcasting
Fixed
Support regulus-usb in getAvailableDevices() on Linux
Fix inferAsync API
Changed
Optimized DMA process on Linux
Don't wait 3 seconds on Model::dispose() without Model::releaseBuffer()
Reduced data copy during inference on Regulus NPU
Refactor infer to resolve reposition-dma bottleneck
v1.2.0
2026-04-02
Added
Support regulus-usb-driver on Linux
Add revision verification for regulus-usb-driver on Linux
Support regulus-usb-driver on Windows
Add resnet50_static.sln for Windows deployment
Support Batch-LLM model
Fixed
Fix getOutputScales to get last bundle's scale, not first
Fix a bug of dumpCacheMemory & loadCacheMemory for primitive LLM Models
v1.1.0
2026-03-23
Added
type: Supports CoreMode::Auto
model: Adds getModelInputDataType & getModelOutputDataType
type: Add getDeviceCount()
Support relocation based dynamic allocation on Regulus
Fixed
model_impl: Fix a bug CoreMode::Auto to select only a single core/cluster on MXQv7
model_impl: Fix a compile error driven by std::filesystem on GCC < 9
npu_task_scheduler: Fix deadlock in activation index allocation for models containing InNPU
relocation_helper: Using move semantic instead of copy
Changed
npu_watchdog: Optimize initRmem to copy all RMEMs at once
reposition: Optimize reposition by MemoryPool
simd_x86_64_scale: Optimize to save SIMD output by _mm256_stream_*
ndarray: Allocate with 32-Byte alignment to optimize reposition
Update supported revision number of regulus-driver from REV0 to REV1
Rename LogLevel enum from UPPER_CASE to PascalCase
v1.0.0
2026-01-29
Added
model.py: Support infer APIs that can specify HWC or CHW
Support relocation based dynamic allocation and 1-core-multi-model runtime
CMakeLists.txt : Add MSVC multi-core build flag
Fixed
CMakeLists.txt : Fix SIMD not applied on Windows
Fix a bug of CacheInfo update failure after launch
Fix issue related to InNPU IOType input
Changed
Rename library from maccel to qbruntime
Removed
model: Remove resetCacheMemory
Remove model claim/unclaim
v0.31.0
2026-01-16
Added
Sync op_desc.cc with unsupported.cc in quantizer
env_var: Support MACCEL_VISIBLE_DEVICES
Add --export-sample-dir option to testinfer
Add driver revision number matching to Regulus
Support UINT8 Models
Fixed
Changed
Modify the build to get the version from the Git tag
Windows: Update build_release.bat and build_wheel.bat
aries: Allow MMIO-based write to NPU SRAM
Supports revised regulus-npu-driver
Removed
Remove documents from maccel repository
Remove build_wheel.sh and build_wheel.bat
Revert "Support multi-batch inference of CPU-Offload models"
v0.30.1
2025-10-20
Added
CI: build & ctest check added
Check MXQ core consistency
Add @note comments for inferAsync family
Show supported cores in error message when ModelConfig mismatches MXQ file
Fixed
Fix issue reading files over 2GB on Windows
npu_watchdog: Fix inference error in activation buffer model with Global-Core
Changed
Supports Windows driver rev#1 : S/G DMA, core claim/unclaim
Removed
Remove the Trace API from acc
v0.30.0
2025-08-29
Added
Supports int8 asynchronous inference
Add getModelSummary function
Send model's memory usage to aries2-driver at Model::launch
Add Python comments for doxygen generation
Support BiRNN, BiLSTM
Fixed
Fix compile error by std::packaged_task in Windows
Fix out of index error of async API unittests
Windows: Restrict each NPU core to load only one model at a time
Fix Already-launched Model error handling
Fix a bug where the variable-length model only accepted a fixed shape
Changed
Update README.md
Change Python API error message
Add numpy dependency to install maccel whl file
v0.29.0
2025-07-25
Added
npu_task_scheduler: Add NPUTaskScheduler
type: Add CacheInfo
Support multi input shape model
Fixed
Fix testinfer.cc help message
model_impl: Fix the logic calculating 2nd activation space when a model has multi-bundle
acc_impl: Block to upload IMEM_INIT/IMEM_INITSTART at Model::launch for Regulus
Fix compiler error caused by designated initializer in Ubuntu18.04
Changed
Enable additional compiler warnings and fix triggered warnings
model: arguments of dumpCacheMemory and loadCacheMemory change from file path to directory path
model: arguments of dumpCacheMemory and loadCacheMemory change from a buffer to buffer vector
v0.28.0
2025-06-25
Added
Support model using framebuffer logic
Support model using command queue inference
Fixed
Fix bugs where models with global-core failed to run when EXPERIMENTAL_READ_ALL_OUTPUTS_AT_ONCE=ON
Changed
model_impl: Use CacheRearrange IMEM in Model::resetCacheMemory
v0.27.0
2025-06-17
Added
Support Cache Rearrange (Tail-Move, Tail-Filter) by Supplementary Infer
Add async API python wrapper
Support cmake --install
Add header installation for cmake --install
Add INSTALL_MACCEL cmake option
Add a post-build event to resnet50.sln to copy a DLL file
Fixed
Fix bugs related to SequenceLength
Changed
Use prettified argv[0] in testinfer help message
Removed
Remove exception handling for dev_no in sleep device
v0.26.0
2025-05-20
Added
Windows: Support building with custom OpenBLAS path
Implement Async API for C++
Support multi-batch inference of CPU-Offload models
Support MXQv5 and models with multiple sequence lengths
Fixed
Resolve build error on Ubuntu 18.04 caused by _mm256_set_m128i
Fix markdown syntax errors and content errors for doxygen documentation
Fix CMake error of add_dependencies during cross-compilation
Fixed error in infer and infer about reversed input shapes
Changed
Improve the Python wheel build process - remove unnecessary logs and ensure rebuilding whl every time
Refactor unittests for the Python API
Update CMAKE_CXX_STANDARD from 14 to 17
reposition_test: Refactor all HWC<->CHW transpose code into unified functions
CMakeLists.txt: Change the default value of MACCEL_CPU_OFFLOAD to bonfire
Improve the execution time of the Model::create
Rename Ticket to Future for consistency with common async terminology
Removed
Remove lib_info.py and resnet50_accuracy.py
v0.25.0
2025-04-03
Added
Support FP16 & FP32 NPU Data Type
testinfer: Support a help message to describe arguments and how to use
Support Global4 and Global8 core modes
pymaccel: Add set_global4_core_mode, set_global8_core_mode, and set_multi_core_mode to python API
Support LLM Model
pymaccel: Add max_height, max_width, max_channel, and max_cache_size in BufferInfo
Support Multi core mode
Add Python unittests to verify the Python API
Apply doxygen to maccel
Fixed
Windows: Fix updating temperature to new union sturcture
Windows: Release pMem after updating memory consumption
tensor_utils: Fix moveToNDArray to execute copy if libtorch is used for CPU offload
Windows: Distinguish the target architecture in .whl file between MinGW Python and native Windows Python
Windows: Fix build errors in the English version of Windows by removing Korean comments from build_wheel.bat and build_release.bat
interleaving: Fix a bug where the return value of interleave was calculated larger than intended
acc_impl: Fix a bug where IMEM is overwritten by IMEM_CACHE_REARRANGE in Model::launch
Fix NDArrayData to have thead-safe refcount by using std::shared_ptr
simd_x86_64_transpose: Add missing defined(__GNUC__) && to prevent compilation in MSVC
npu_op_desc_test: Fix the failure of NPUOpDescTest.getRmemAddresses in Aries1
pymaccel: Fix get_latency_set_policy being overlapped by get_latency_consumed
testinfer: Fix set_global8_core_mode being overlapped by set_global4_core_mode in Python
Fix the logic for calculating DDR memory usage to avoid referencing hwdep and to support multi-core mode
Windows: Fix to pass the correct Aries1 DDR usage to Windows monitoring tools
Fix Python APIs - add __repr__, is None, reposition_outputs, np.ascontiguousarray, and more
test_model: Fix test_checkInferConsistency to correct comparison
Changed
Windows: power monitor change to power/voltage/current
driver: Remove the 4KB margin in allocHostMemory due to an interleaving bug
pymaccel: Change the notation of some methods in ModelConfig from camelCase to snake_case
Windows: Update Windows codes to match Windows driver v1.6
ndarray: Attach noexcept to move constructor/assignment of NDArrayData
Make ModelConfig API more user-friendly - Add setSingleCoreMode
Revise the Python API wrapper of maccel
Improve docs - Markdown formatting, Python ModelConfig example in advanced_usage, and more
Removed
model_impl: Remove redundant trace events in inferBufferOutput and inferSpeedrun
Remove packed logic
v0.24.0
2024-02-12
Added
Windows: Tracks memory usage of Aries and sends it to the Windows monitoring tool
Fixed
Modify the naming of .whl files to properly reflect the target platform, such as Windows or Linux
v0.23.0
2024-01-24
Added
type.h: Add option in ModelConfig to force a single NPU bundle execution
Support FP16, FP32 Output NPU Buffer
Fixed
type.h: Support Global core mode
v0.22.0
2024-01-23
Added
Support Aries2 on maccel
Windows: Add aries performance monitoring
Fixed
Fix build error on Windows by using findPythonInterp in MSVC
Changed
Windows: Send 8->1->0 in postInfer on Windows
v0.21.0
2025-01-17
Added
init.cc: Add initializer to set default LogLevel according to MACCEL_LOG_LEVEL
model: Add shape parameter to std::vector<T*> infer api
Support RNN/LSTM models with fixed sequence length inputs
testinfer: Add seq-sizes argument for variable length inputs
CMakeLists.txt: Automatically generate libmaccel.so* symbolic links based on the SOVERSION and VERSION extracted from the Git tag
Windows: Support Aries1/2 on Windows
Fixed
reposition: fix logic of need_repos when shape is kept but reposition occurs
Modify the naming of .so and .whl files to appropriately reflect the target architecture
Fix whl file name error by adding linux_ prefix
Changed
Support multi-model on single NPU core for Regulus
acc: Revise Accelerator::getCoreList to retrieve all available cores from driver
regulus: Fix regulus timeout unit from nsec to mesc
CMakeLists.txt: Add a cmake flag MACCEL_GLIBCXX_DEBUG to apply compile option D_GLIBCXX_DEBUG
npu_watchdog: dumpDDRBin to dump DDR for each bundle & sequence_index
CMakeLists.txt: Remove Warning from FindPython
v0.20.0
2024-11-15
Added
op_desc.cc: Add support for additional CPU offload operations
type: Add CoreAllocationPolicy to support automatic NPU core allocation
Fixed
CMakeLists.txt: Consider CMAKE_BUILD_TYPE as Release, when it is not specified
Changed
op_desc.cc: Update implementations for some of CPU offload operations
CMakeLists.txt: CMAKE_BUILD_TYPE=Release limits LogLevel to INFO
v0.19.0
2024-09-12
Added
simd_x86_64_scale: Implement SIMD(AVX2, SSE2)-based scale functions in x86-64 (#432, #434)
simd_x86_64_transpose: Implement SIMD(AVX2, SSE2)-based transpose functions in x86-64 (#434)
transpose: Implement transpose functions for inferCHW (#434)
simd_aarch64_scale: Implement SIMD(NEON)-based scale functions in ARM64 (#435)
simd_aarch64_transpose: Implement SIMD(NEON)-based transpose functions in ARM64 (#435)
Add definitions of platform which represents host's architecture & SIMD (#441)
aries_win: Implement postInfer for Windows (#446)
Support multi-card in Windows (#448, #450)
Support Regulus NPU (#449)
model_impl : Implement runNPUModelTest for use in inferSpeedrun and inferOutputDiff of Thread-Benchmark (#462)
Support MXQv3
Fixed
reposition: Fix calcReposIndicesDefaultBase when original_size != reshaped_size (#437)
reposition_test: Fix vector out of bound in scale_list (#438)
Update SIMD Option for MSVC (#441)
Fix some unittests to support MSVC (#446)
win_ddk: Fix pure function for MSVC (#453)
Changed
reposition: Use SIMD for float repositions (#432, #434, #435)
PCIeDriver: Exclude RiscV area in Windows heap allocator (#443)
reposition: Use SIMD-based transpose for CHW int8_t data (#442)
npu_watchdog: Unify postInfer in both Windows and Linux (#446)
reposition: MXQ compiled by default use efficient reposition (#457)
reposition: repositionFloat & repositionInt are integrated (#460)
Removed
reposition: Remove OpenMP (#447)
reposition: Remove default reposition (#457)
v0.18.0
2024-04-30
Changed
reposition: Revise need_repos condition to compare size of original and buffer (#431)
v0.17.0
2024-04-12
Added
testinfer: Add batch-size, num-cores options (#414)
pymaccel: Add set_log_level, start_tracing_events, stop_tracing_events (#418)
pymaccel: Implement maccel.load() API in Python (#422)
model: Implement acquireInputBuffers, acquireOutputputBuffers, releaseBuffers (#428)
model: Implement new infer API return multi-batch output by reference (#428)
model: Implement new repositionOutputs which use vector<vector<float>>& output (#428)
model: Implement inferBufferToFloat (#428)
Fixed
npu_watchdog: Fix default time duration overflow (#425, #426)
resnet50_test_cc: Fix wrong comparison & scale and add diff test (#428)
reposition: Fix fillgap logic (#429)
Changed
aries: Determine to use interrupt mode by ARIES_IOC_GET_SIGNAL_TYPE ioctl rather than driver version (#417)
model_impl: Skip benchmark for re-launched model (#421)
npu_watchdog: Change Default timeout from 1s to 10s (#424)
reposition: Use scale.scale when scale is uniform (#420)
v0.16.0
2024-02-20
Added
build_wheel: MSVC provides python wheel (#409)
Fixed
Fix pymaccel build error from latest setuptools version (#412)
reposition: Fix ch-wise scale bug in inferCHW (#415)
Fix input shape check for batch in Python (#416)
Changed
aries: Enhance robustness of pread/pwrite usage by apply loop (#413)
Removed
resnet50_msvc: Remove .TestDrive, PCIeDriverSystem.dll (#409)
VERSION: Remove VERSION (#412)
v0.15.0
2024-01-03
Added
testinfer: Implement inferBuffer infer-api
acc_impl: Add lock_guard for AcceleratorImpl
pymaccel: Implement constructor that create and launch at the same time in Python (#391)
model_impl, npu_watchdog: Implement IMEM_INIT, IMEM_INITSTART
Fixed
memory_pool: Fix not to wait forever when model.dispose() called without releaseBuffer
reposition: Apply __pragma for MSVC (#393)
aries_win: Addtional allocate 4KB host memory to fix interleaving bug (#396)
Fix default log level bug in release build
model_impl: Fix segfault in user_outputs by rollback of resize removal (#401)
Changed
testinfer: Modularize doMain by implementing processInputs, processOutputs, printSummary
model_desc: Revise model shape to exclude batch-size (#392)
model_desc: checkIfInputShapeMatchAndEnsureBatchDim -> doesInputShapeMatch (#392)
op_desc: Update CPU offload code (#386)
tensor_utils: Remove release when move tensor to NDArray
Change PACKAGE_FILENAME as maccel_${VENDOR}_${PRODUCT}_${VERSION}
v0.14.0
2023-11-03
Added
pypymaccel (#369)
acc: Introduce startTracingEvents(), while deprecating Acc::startTracingEvents()
model: Implement infer APIs using NDArray
Fixed
reposition: Calulcate CHW indices for efficient type
acc_impl: Respect OMP_NUM_THREADS env var (#378)
driver: Check {read,write}MemoryBuffer's return value (#370)
Changed
omp: Limit number of threads for OMP parallel block when other OMP blocks are running (#383)
윈도우에서 아래 파일들의 추가 종속성을 없앰 .TestDrive SystemDDK.dll SystemHAL.dll PCIeDriverSystem.dll
Convert maccel.h to all-in-one-style header (#366)
type: Include all cluster/cores by default
v0.13.0
2023-09-13
Added
Add copyright notice (#351)
npu_watchdog: Dump DDR.bin for debugging (#348)
Fixed
reposition: Fill 0 when reposition is ill-fitted
reposition: Fix filling gap bug for channel 1
Changed
cmake: Add 'd' postfix for windows dll debug build
log: Change default log level
model: Implement inferHeightBatch using inferBatch (#349)
v0.12.0
2023-09-01
Added
aries: Introduce new env var, which prevents {un,}claiming cores (#338)
Interrupt 지원 (Linux) (#324)
windows_interrupt (#315)
INFO, WARNING, ERROR 로그가 출력됩니다.
추가된 setLogLevel() API로 로그 레벨을 조절할 수 있습니다. (OFF도 가능)
Implement int8->int8 infer API in output-as-parameter fashion
Fixed
reposition: Fill gap when input channel is 1
reposition: Don't use efficient reposition if it's CHW format
Fix Visual Studio build error (#322)
reposition: Make output correct for AVX2
memory_pool: When reset, free all pre-allocated memory
Changed
reposition: Implement efficient logic for int input/output (#329)
Limit OMP num threads at runtime
Implement RepositionType::EfficientRuntimeInterleave
Removed
model: Remove some infer APIs
v0.11.1
2023-08-25
aries: Introduce new env var, which prevents {un,}claiming cores (#339)
v0.11.0
2023-06-21
Added
Implement experimental inferHeightBatch() API
model: Implement new infer API which takes output as parameter
Fixed
Fix inferSpeedrun segfault
reposition: When CHW, use default method
Changed
reposition: Implement scale using NEON
Refactor reposition functions
v0.10.0
2023-06-14
Added
Implement batch input inference API (#290)
Fixed
Fix EXPERIMENTAL_READ_OUTPUTS_AT_ONCE compile error
Fix inferCHW bug
Mitigate sleep() problem on windows (#283)
Changed
Do not repos when original channel is the same as pe num (#294)
Allocate input/output buffers at once (#293)
Implement RepositionType::Efficient (#291)
pymaccel check input shape (#292)
pybind11 infer no convert (#289)
Implement MemoryPool and apply to ModelImpl (#288)
Move NPUTaskQueue-releated members to AcceleratorImpl (#286)
reposition: Clean up omp directives
Remove model runner
v0.9.1
2023-05-23
Changed
v0.9.0
2023-05-19
Added
Implement manual packed, multi input inference
Claim cores when model launches (#277)
Changed
List up more detailed items for profiler (#278)
kibum/factor-out-aries-reset (#276)
Try to apply bonfire for CPU offload (#273)
root일때만 whl 빌드 (#274)
v0.8.0
2023-04-18
Fixed
Use input's memory format for output's memory format (#270)
Changed
Support width-wise reshape with 1 or 2 channels, just like 3 channels (#272)
Build a wheel file while makeing
Allow lowercase for DRIVER_TYPE
0.7.0
2023-03-23
Added
Initial work on CPU-offloading.
Removed
Temporarily removed non-float infer APIs
0.6.0
2023-03-23
Added
add windows driver interface. (tested.) (#263)
Implement GlobalMode/MultiMode (#262)
core 관련 함수 추가 (#251)
add modelconfig func (#230)
Reset 구현 (#229)
Implement inferSpeedrun (#232)
model: Add infer functions using int8_t (#182)
version 기능 추가 (#173)
model_impl: Implement ChannelFirst (#166)
Changed
Plus 0.5 then floor to convert float -> int
npu_watchdog: Set reset timeout ratio to 10 (#241)
Enable log for default cmake build
pymaccel infer gil release 추가 (#214)
profile: Reimplement profiler (#210)
Aries 이후 하드웨어를 지원하는 패치 (#185)
model_manager: Always wait until NPU_FINISH == 1
Fixed
Fix generate export header process
acc 먼저 파괴시 model dispose 호출 (#252)
동작중인 Core에 launch 방지 (#211)
모델 업로드 전 DRAM 영역 초기화 로직 추가 (#206)
Markdown llms.txt
맨위로