p13709645
/
InfiniTensor
forked from jiuyuan/InfiniTensor
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
186 Commits 70 Branches 0 Tags 141 MiB
Commit Graph

186 Commits

Author SHA1 Message Date
whjthu 34ed298725 fix format 2023-04-22 17:00:52 +08:00
whjthu 664f0dbe02 support cuda transpose 2023-04-22 16:57:27 +08:00
whjthu 225a42f22d add rule for dilated conv 2023-04-21 23:40:45 +08:00
Liyan Zheng d051460c23 Chore: suppress output 2023-04-21 22:58:18 +08:00
Liyan Zheng d8a133684e Add: remove independent tensors in graph 2023-04-21 22:57:23 +08:00
Liyan Zheng 9ce21200c4 Add: NMutator mode in python 2023-04-21 21:31:22 +08:00
Liyan Zheng b943658713 Finish: GAN 2023-04-21 21:25:43 +08:00
Liyan Zheng 2cd75bd79b Merge branch 'NNET_e2e_fix' into NNET_e2e 
Support CUDA Graph for TVM kernels
 2023-04-21 13:18:44 +08:00
Liyan Zheng f0fcbe825f Add: python verification 2023-04-21 13:18:24 +08:00
huangshuhong 8c91faa948 remove expect 2023-04-21 00:17:04 +08:00
huangshuhong c0ae03a2d7 fix tvm stream 2023-04-21 00:09:47 +08:00
Liyan Zheng 0cb8729bc1 Add: different ONNX names for inputs and weights 2023-04-20 21:51:47 +08:00
YdrMaster 8bc2d3e48d fix: test graph handler 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
 2023-04-20 21:51:47 +08:00
YdrMaster 28b123753e feat: 导入 Tensor 类型 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
 2023-04-20 21:51:47 +08:00
Liyan Zheng 94730d93b5 Add: hash match for membound kernels 2023-04-20 17:16:01 +08:00
Liyan Zheng 6d17c4caa2 Add: getPerfTime in run_models_nnet 2023-04-20 10:54:49 +08:00
Liyan Zheng 15d0eb79cd Add: import ONNX with membound Op 2023-04-20 10:45:28 +08:00
Liyan Zheng 2a343e240e Add: shape of intermediate tensor in exported ONNX 2023-04-20 10:45:28 +08:00
Liyan Zheng 34ca6bf149 Fix: skip check when Graph is exported to ONNX 2023-04-20 10:45:28 +08:00
YdrMaster a6019e79e3 feat(py): 支持从 Graph 直接创建 OnnxStub 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
 2023-04-20 10:45:28 +08:00
YdrMaster 4e1cc8d3e4 refactor(py): 使用工厂方法创建 OnnxStub 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
 2023-04-20 10:44:39 +08:00
YdrMaster 725f9260cf feat: 支持导出 membound 
Signed-off-by: YdrMaster <ydrml@hotmail.com>
 2023-04-20 10:44:39 +08:00
YdrMaster 0edd138919 feat: 正反序列化分离为到 string 的和到 file 的 
fix: 正确设置 `USE_CUDA` cfg

todo: test_search 不过

Signed-off-by: YdrMaster <ydrml@hotmail.com>
 2023-04-20 10:44:39 +08:00
Liyan Zheng 0b23a065ca Add: debug hacks for InfoGAN 2023-04-20 10:42:56 +08:00
Liyan Zheng e86e993ed4 Add: CUDA graph stream capture (MemboundOp fails) 2023-04-19 16:32:16 +08:00
Liyan Zheng e4c20a9ae2 Add: warmup and repeat args in timeNonCtcOperators 2023-04-19 16:22:59 +08:00
Liyan Zheng 537b3b4ea4 Add: Membound operator serialization 2023-04-18 21:53:48 +08:00
Liyan Zheng 2812900ea2 Fix: OpType and print device tensors 2023-04-18 20:28:08 +08:00
Liyan Zheng 01fc19795d Add: time non-compile-cime-computable operators 2023-04-18 17:21:16 +08:00
Liyan Zheng afc4123328 Chore: remove deprecated function 2023-04-18 17:21:16 +08:00
Liyan Zheng b981951a47 Add: NMutator::memboundToJson to export memboundOp 2023-04-18 17:21:16 +08:00
Liyan Zheng 99b5c95455 Add: nnet::Serializer supports FuncNode 2023-04-18 17:21:16 +08:00
Liyan Zheng 9d50b30af8 Chore: disable nnet_unimplemented_continue output 2023-04-18 17:21:16 +08:00
Liyan Zheng bc31219bde Add: exclude compile-time computable operator time 2023-04-18 17:21:16 +08:00
Liyan Zheng edf4e33353 Add: C++ callback to export ONNX 2023-04-18 17:19:05 +08:00
Liyan Zheng 872f3504a9 Add: RangeOpNode::getFullExpression() 2023-04-18 17:19:05 +08:00
Liyan Zheng da49e91ab0 Add: fuse membound operators 2023-04-18 17:19:05 +08:00
Liyan Zheng a6b8f344d4 Chore: simplify type names 2023-04-18 17:19:05 +08:00
Liyan Zheng 09293730ea Add: export to ONNX with custom operators 2023-04-18 17:19:05 +08:00
Liyan Zheng 307614d95d Add: infogan python interface 2023-04-18 17:16:25 +08:00
Liyan Zheng f14edcd52f Fix: avoid reload library 2023-04-18 17:16:25 +08:00
Liyan Zheng d2d49c5d4f Add: invoke TVM through pipe 2023-04-18 17:16:25 +08:00
Liyan Zheng e72fe79168 Add: search engine uses estimated time 2023-04-18 17:16:25 +08:00
YdrMaster 26f0d13c26
Dev for 202303ddl (#66) 
* add activation operatiopn relu, tanh, sigmoid on mlu

* commit for format

* add activation backward operation

* add test for activation_backward

* add test

* add convbpfilter

* fix

* add transpsoe code and test

* add trigon function operation on mlu: sin,cos,tan,asin,sinh,asinh

* add copy operation on mlu

* add ceil operation and floor operation

* add operation clip

* add operation cnnl div, test and test for divdemo bangc kernel

* add divnonan operation and test

* add erf operation

* add exp operation

* add operation fill

* add log operation

* add log1p operation

* add l2loss operation

* add maximum and minimum operation

* add mseloss operation

* add negTensor operation

* add power operation

* add reciprocal operation

* add sqrt and rsqrt operation

* add transform operation

* add addn operation

* add muln operation

* cherrry pick some operation

* add floordiv operation and floordivtrunc operation

* add floormod operation

* add cumsum operation

* add det operation

* add pad operation

* format

* add concat operation

* format

* add split operation

* fix concat and split operation

* add round operation

* add pooling operation

* add square operation

* add squaredDifference operation

* code format fix

* add flip operation

* code format fix

* add hardtanh operation

* add logic operation

* add addcdiv and addcmul operation

* add arange operation

* add bitcompute operation

* add net test

* fmt

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* style: rename

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: 用 NativeCpuRuntime 替换 CpuRuntime

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix code

* fix code

* fix code by review suggestion

* remove operation which is not the onnx operation

* fix format

* clang format

* refactor: tensor 的 print 加一层模板的 dataToString

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: onnx 导出

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 增加计算图优化接口

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* add clip operation

* feat: 支持导入 clip

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* test: 导入导出测试加入 ci

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix batch norm

* feat: 增加 Shape 算子

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 支持导入 unsqueeze

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: 修正 clip 接口

feat: 支持导入 transpose
Signed-off-by: YdrMaster <ydrml@hotmail.com>

* add broadcast operation

* fix elementwise-broadcast

* fix elementwise broadcast

* add broadcast for gpu elementsie

* feat: pad 支持 axes 负数

feat: 不支持的 padding 导出为独立的 pad 算子

feat: 支持导入 onnxsim 过的 inception
Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: 修正池化的测试

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 导出 pads，支持 inception 导入导出，已加入 ci

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 支持 densenet 导入导出，并加入 ci

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 导入 squeeze

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix softmax

* feat: 导出 clip 和 transpose

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 支持 Conv 的 bias

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: bias of conv

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: bias of conv

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 导入 split

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 导出 split

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: conv

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: conv group

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: matmul 的 bias 没有放在输入里，修正

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix exmaple

* fix: 改正 reduce_mean 导出

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* refactor: 修改 slice 实现与 onnx 一致

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* style: 不导出两个 runtime 函数

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* doc: 中文使用指南

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* doc: 补全指南

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: 修复导入数据的问题

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fmt

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 添加 Dropout 基本结构，但不支持两个输出是不同的类型

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 重新导出优化接口

feat: dropout 导入
Signed-off-by: YdrMaster <ydrml@hotmail.com>

* build: BANG 选项加入 Makefile

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fxi code, change of test/kernels/bang/test* is use NativeCpuRuntime.
chaneg of include/bang/bang_runtime is for the cntoolkit upgrade.

* feat: 导出 bang runtime

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* add USE_BANG=1

* fix matmul

* fix reshape

* fix

* fix activation

* fix transpose

* format

* format

* update Makefile

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 支持导入导出 ConvTranspose

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* add prelu on mlu

* fix: ConvTranspose

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* feat: 支持导入导出 PRelu

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* add convtrans on mlu

* fmt

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* docs: 更新 README_CN.md

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix code by review suggestions

* style

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix: Softmax 的 axis 可以用默认值？感觉是 onnx 不标准

Signed-off-by: YdrMaster <ydrml@hotmail.com>

* fix cuda & intelcpu bugs after merging

---------

Signed-off-by: YdrMaster <ydrml@hotmail.com>
Co-authored-by: wanghailu <wanghailu0717@163.com>
Co-authored-by: wanghailu <wanghailu@qiyuanlab.com>
Co-authored-by: whjthu <haojie0429@gmail.com>
 2023-04-18 15:10:33 +08:00
zhengly123 a1974aabcd
NNET supports TVM backend and kernels (#78) 
* Add: mutator InfoGAN minimum test

* Add: cache and padding (bugs!!)

* Add: expression reader as a cmake target

* Fix: [Intermediate] NMutator::expressionToGraph

To be fix: matmul with implicit broadcast

* Add: matmul broadcast

* Fix: GraphObj ctor should use cloneTensor

* Fix: cuBLAS failure when codegen is enabled

* Add: Exception for checkCuError

* Fix: graph OpList ctor

* Add: expr simplication for TVM

* Add: TVM headers and CMake include paths

* Add: CMake config

* Add: PackedFunc (broken)

* Fix: remove cuCtxCreate which makes TVM fails

* Fix: membound_tvm

* Fix: test_memboundOp

* Add: PRelu Expr and AsTVMVisitor

* Add: Random generator

* Add: support TVM packed function

* Fix: specify runtime

* Add: CMake support of TVM

* Add: detailed output of Matmul

* Add: comments for Matmul

* Chore: format and comments

* Chore: GraphObj::selfCheck without assert control

* Fix: CMAKE_CXX_FLAGS in CMakeLists

* fix merge bug

* update api for mkl batchnorm test

* fix lotus env

* fig header bug

---------

Co-authored-by: Liyan Zheng <liyan-zheng@outlook.com>
Co-authored-by: huangshuhong <huangsh19@mails.tsinghua.edu.cn>
Co-authored-by: whjthu <haojie0429@gmail.com>
 2023-04-18 00:26:36 +08:00
wendy12022 43d4798323
ADD: sub graph replacement. (#56) 
reconfig: connections among op and tensor now is managered by GraphObj .

add some comments

merge from master

merge from master

ADD: sub graph replacement

reconfig inputs of op resize, due to the check of operator inputs.

ResizeObj::clone

clang format

fix some and add test for multi-output.

replacement support multi-inputs and multi-outputs.

add clone for all operators

add replaceSubGraph addSubGraph

remove extra code

add more test

remove extra print

Co-authored-by: Haojie Wang <haojie0429@gmail.com>
 2023-04-17 13:09:07 +08:00
wendy12022 c8b2c8ed32
Cpu backend2 (#77) 
fix review

change Device::MKL to Device::INTELCPU

fix mkl linkage

fix errors according to merge from master

now can call mkl backend

fix softmax/flatten with axis from onnx.

modify README.md

fix memory refree

add env_lotus_intelcpu.sh

fix compile

merge from branch cpu_backend

fix something add gather

fix something

FIX: directory rename from "mkl" to "intelcpu"

ADD: use oneMKL dpcpp interface to implement matmul kernel.

ADD: add dpcpp as compiler for mkl, and fix warnings for clang compiling.
add dpcpp kernel for pow.

ADD: mkl kernel for pad.

ADD: slice mkl kernel.

ADD: reshape/flatten/identity mkl kernel.

ADD: split mkl kernel.

fix compile error

FIX: fix flattenObj with axis.

ADD reduce_mean mkl kernel.

Add concat mkl kernel.

bathNorm for mkl kernel.

sigmoid mkl kernel.

ADD：add mkl kernel for pooling

add more tests for softmax

Now softmax cuda kernel supports any axises.

mkl kernel for softmax

softmax

add axis to softmax operator

add mkl kernel for abs tanh

ADD: relu kernel for mkl

fix binary mkl primitives.

add mkl kernel for binary operators

fix compiler error

move stream to runtime

clang format

add MemoryFormat for tensorObj.

use post_ops for fused conv/deconv

Distinguish mkl  op_timer from cuda op timer.

add act optype to conv and deconv

add operator timer

add mkl kernel for convTransposed

minor fix for group conv

do not use cblas_sgemm_batch

CpuRuntimeObj->NativeCpuRuntimeObj

add  matmul op for mkl
 2023-04-17 12:15:23 +08:00
Hardy fe1afe38fa
fix code of bang conv (#76) 
* fix code of bang conv

* test: 向 master push 时也执行 ci

Signed-off-by: YdrMaster <ydrml@hotmail.com>

---------

Signed-off-by: YdrMaster <ydrml@hotmail.com>
Co-authored-by: wanghailu <wanghailu@qiyuanlab.com>
Co-authored-by: YdrMaster <ydrml@hotmail.com>
 2023-03-29 15:47:32 +08:00
Hardy 823e66a9ff
Support perf bang 1115 (#57) 
* support matmul

* add matmul

* add matmul

* add code for cnnl matmul operation and test

* add conv

* add code for conv test on mlu

* add code for test cnnl conv on mlu

* add code for perf conv and matmul on mlu

* clang format

* fix convolution operation

* fxi cmaklist

* code format

* fix code

* code format

---------

Co-authored-by: wanghailu <wanghailu@qiyuanlab.com>
Co-authored-by: wanghailu <wanghailu0717@163.com>
 2023-03-29 13:52:56 +08:00
wendy12022 86ec4036ce
ADD: add mkl runtime for intel cpu , and add mkl kernel for matmul/conv/convtransposed. (#61) 
* move memory format transformation to TensorObj

clang format

add MemoryFormat for tensorObj.

use post_ops for fused conv/deconv

Distinguish mkl  op_timer from cuda op timer.

add act optype to conv and deconv

add operator timer

add mkl kernel for convTransposed

minor fix for group conv

do not use cblas_sgemm_batch

CpuRuntimeObj->NativeCpuRuntimeObj

add  matmul op for mkl

* fix: fix bugs when rebasing from master

fix: fix bugs when rebasing from master

* fix: update api after rebasing

* fix: fix format; fix onnx import

* fix: fix clang-format

* [fix] fix conv_transpose test

* [fix] use stronger test case for transposed conv

* [fix] remove tensor memory format; fix mkl transpose conv

* [fix] add FIXME tag for op_timer python api

---------

Co-authored-by: whjthu <haojie0429@gmail.com>
 2023-03-27 21:28:49 +08:00