commit 27d5499f8ad65b0770fd3c9a5d41e5b778a0aa82 Author: nezharen Date: Wed Apr 10 11:14:51 2024 +0800 init diff --git a/ProjectDeploy.md b/ProjectDeploy.md new file mode 100644 index 0000000..838407a --- /dev/null +++ b/ProjectDeploy.md @@ -0,0 +1,1125 @@ + +# 大数据智能分析系统部署文档20240410 + +## 大数据智能分析系统介绍 + +大数据智能分析系统在Apache Spark的基础上构建了一套以性能为中心的高效数据处理系统。 +一方面实现了一套基于C++的RDD编程接口和高性能大数据算子库以支持高性能大数据应用的编写； +另一方面提出了动静态结合的计算图使得在容错能力上等同Apache Spark。 +同时，在数据预处理、图计算、机器学习等领域典型应用上进行了测试，取得了多个指标的性能优势。 +该系统目前已支持TPC-DS中所有99个查询，且在相同环境下相比Apache Spark SQL达到了3.0倍以上的加速比。 +该系统可以降低大数据处理的整体延迟、并可节省大数据处理的成本； +同时能够无缝部署在现有Apache Spark环境中，部署简单。因此，智能大数据分析系统具有显著的应用前景。 + +## 验证环境 + +此文档在6节点集群环境下验证，每台节点的环境如下： + +| 条目 | 配置 | +| ------ |--------------------------------------------------| +| 服务器型号 | 浪潮NF5280M6 | +| 处理器 | Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz * 2 | +| 内存 | 3200 MT/s 64 GB DDR4 内存 * 16 | +| 硬盘 | NVMe 7TB 固态硬盘 * 8 | +| 网卡 | Mellanox Technologies MT28908 Family | +| 操作系统 | Ubuntu 22.04.2 LTS | +| 内核 | Linux 5.15.0-60 | +| C++编译器 | Clang 16.0.4, GCC 10.2.0 | +| Java | Oracle JDK 11.0.17 & 8u202 | +| Spark | Apache Spark 3.1.2 | +| HDFS & Yarn | Apache Hadoop 3.3.6 | +| Hive | Apache Hive 3.1.3 | + +## 如何部署和运行大数据智能分析系统 + +此文档主要提供了搭建大数据智能分析系统jydata运行环境、搭建TPC-DS测试环境、基于大数据智能分析系统运行TPC-DS查询并进行性能测试等相关内容。 +以下为本文档的默认目录结构： +源代码目录：```~/jydata/sources``` +项目的环境变量设置脚本：``~/jydata/setenv.sh`` +依赖软件安装目录：```~/jydata/local``` +第三方软件包目录：``~/jydata/third-party`` +可执行脚本目录：``~/jydata/sources/CMR/sbin`` +本文档默认集群包含``node01``，``node02``，……，``node06``共6个节点，其中``node01``为大数据智能分析系统运行的Driver所在节点。 +如您的实际集群环境不同，请参考本文档根据您的集群情况作适当调整。 + +首先将本项目压缩包在集群所有节点解压缩到``~``目录下： + +x86_64环境： + +```shell +jydata@node01~node06$ tar -zxvf jydata_release_YYYY_MM_DD_x64.tar.gz -C ~ +jydata@node01~node06$ mv ~/jydata_release_YYYY_MM_DD_x64 ~/jydata +``` + +aarch64环境： + +```shell +jydata@node01~node06$ tar -zxvf jydata_release_YYYY_MM_DD_aarch64.tar.gz -C ~ +jydata@node01~node06$ mv ~/jydata_release_YYYY_MM_DD_aarch64 ~/jydata +``` + +## 环境变量 + +```shell +# 将如下内容写入到你的 bashrc 文件并 source 该文件 +export SPARK_HOME=$HOME/jydata/local/spark-3.1.2 +export PATH="$SPARK_HOME/bin:$PATH" +export JAVA_HOME=$HOME/jydata/local/jdk-11.0.17 +export PATH="$JAVA_HOME/bin:$PATH" +export GCC_10_HOME=$HOME/jydata/local/gcc-10.2.0 +export PATH="$GCC_10_HOME/bin:$PATH" +export PATH="$HOME/jydata/local/cmake-3.26.5/bin:$PATH" +export LLVM_16_HOME=$HOME/jydata/local/llvm-16.0.4 +export CC="$LLVM_16_HOME/bin/clang" +export CXX="$LLVM_16_HOME/bin/clang++" +export PATH="$LLVM_16_HOME/bin:$PATH" +export HADOOP_HOME="$HOME/jydata/local/hadoop-3.3.6" +export HADOOP_CONF_DIR="$HADOOP_HOME/etc/hadoop" +export PATH="$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH" +export HIVE_HOME="$HOME/jydata/local/apache-hive-3.1.3-bin" +export PATH="$HIVE_HOME/bin:$PATH" + +# 如您不方便将上述环境变量配置到 bashrc 文件中进行长期使用，你也可以在我们提供的 setenv.sh 文件中进行正确配置并激活，作为临时使用 +jydata@node01$ source ~/jydata/setenv.sh +``` + +## 运行环境搭建部署 + +### 1. Spark部署 + +下载并解压Spark 3.1.2到``~/jydata/local``目录下。 + +```shell +jydata@node01$ tar -xvf spark-3.1.2-bin-hadoop3.2.tgz -C ~/jydata/local +jydata@node01$ cd ~/jydata/local +jydata@node01$ mv spark-3.1.2-bin-hadoop3.2 spark-3.1.2 +``` + +``~/jydata/setenv.sh``为项目的环境变量设置脚本。接下来设置并应用Spark相关的环境变量： + +```shell +jydata@node01$ echo 'export SPARK_HOME=$HOME/jydata/local/spark-3.1.2' >> ~/jydata/setenv.sh +jydata@node01$ echo 'export PATH="$SPARK_HOME/bin:$PATH"' >> ~/jydata/setenv.sh +jydata@node01$ source ~/jydata/setenv.sh +``` + +然后在``node01`` ~ ``node06``每个节点下载并安装Spark和jydata的依赖，Java JDK 11.0.17。 + +x86_64环境： + +```shell +jydata@node01~node06$ tar -zxvf jdk-11.0.17_linux-x64_bin.tar.gz -C ~/jydata/local +``` + +aarch64环境： + +```shell +jydata@node01~node06$ tar -zxvf jdk-11.0.17_linux-aarch64_bin.tar.gz -C ~/jydata/local +``` + +在``node01``设置并应用JDK相关的环境变量： + +```shell +jydata@node01$ echo 'export JAVA_HOME=$HOME/jydata/local/jdk-11.0.17' >> ~/jydata/setenv.sh +jydata@node01$ echo 'export PATH="$JAVA_HOME/bin:$PATH"' >> ~/jydata/setenv.sh +jydata@node01$ source ~/jydata/setenv.sh +# 查看JDK版本 +jydata@node01$ java -version +# 样例输出 +java version "11.0.17" 2022-10-18 LTS +Java(TM) SE Runtime Environment 18.9 (build 11.0.17+10-LTS-269) +Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.17+10-LTS-269, mixed mode) +``` + +### 2. Lancher + +在``~/jydata/sources/CMR/cmr-spark-adapter/target/scala-2.12``目录下，已提供编译好的``cmr-spark-adapter_2.12-0.5.1.jar``文件。 + +### 3. 下载并安装依赖 + +#### 3.1 GCC编译安装10.2.0版本 + +```shell +jydata@node01$ mkdir ~/jydata/local/gcc-source +jydata@node01$ tar -zxvf gcc-10.2.0.tar.gz -C ~/jydata/local/gcc-source/ +# 设置GCC的编译依赖gmp、mpfr、mpc +jydata@node01$ tar -xvf gmp-6.1.2.tar -C ~/jydata/local/gcc-source/gcc-10.2.0 +jydata@node01$ ln -s ~/jydata/local/gcc-source/gcc-10.2.0/gmp-6.1.2 ~/jydata/local/gcc-source/gcc-10.2.0/gmp +jydata@node01$ tar -zxvf mpfr-4.2.0.tar.gz -C ~/jydata/local/gcc-source/gcc-10.2.0/ +jydata@node01$ ln -s ~/jydata/local/gcc-source/gcc-10.2.0/mpfr-4.2.0 ~/jydata/local/gcc-source/gcc-10.2.0/mpfr +jydata@node01$ tar -xvf mpc-1.3.1.tar.gz -C ~/jydata/local/gcc-source/gcc-10.2.0/ +jydata@node01$ ln -s ~/jydata/local/gcc-source/gcc-10.2.0/mpc-1.3.1 ~/jydata/local/gcc-source/gcc-10.2.0/mpc +# 编译安装GCC +jydata@node01$ mkdir ~/jydata/local/gcc-10.2.0 +jydata@node01$ cd ~/jydata/local/gcc-source/gcc-10.2.0 +jydata@node01$ patch -p0 < ~/jydata/third-party/patches/libstdc++-v3_Makefile.in.patch # 修复编译可能出现的问题 +jydata@node01$ patch -p0 < ~/jydata/third-party/patches/libstdc++-v3_Makefile.am.patch # 修复编译可能出现的问题 +jydata@node01$ mkdir build && cd build +jydata@node01$ ../configure --prefix=$HOME/jydata/local/gcc-10.2.0 --enable-checking=release --enable-languages=c,c++ --disable-multilib +jydata@node01$ make -j 32 # make并发数可根据实际情况调整 +jydata@node01$ make install +# 添加相关环境变量 +jydata@node01$ echo 'export GCC_10_HOME=$HOME/jydata/local/gcc-10.2.0' >> ~/jydata/setenv.sh +jydata@node01$ echo 'export PATH="$GCC_10_HOME/bin:$PATH"' >> ~/jydata/setenv.sh +jydata@node01$ source ~/jydata/setenv.sh +# 查看GCC版本 +jydata@node01$ gcc --version +# 样例输出 +gcc (GCC) 10.2.0 +Copyright (C) 2020 Free Software Foundation, Inc. +This is free software; see the source for copying conditions. There is NO +warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + ``` + +#### 3.2 cmake 安装3.25以上版本 + +x86_64环境： + +```shell +jydata@node01$ tar -zxvf cmake-3.26.5-linux-x86_64.tar.gz -C ~/jydata/local +jydata@node01$ cd ~/jydata/local +jydata@node01$ mv cmake-3.26.5-linux-x86_64 cmake-3.26.5 +``` + +aarch64环境： + +```shell +jydata@node01$ tar -zxvf cmake-3.26.5-linux-aarch64.tar.gz -C ~/jydata/local +jydata@node01$ cd ~/jydata/local +jydata@node01$ mv cmake-3.26.5-linux-aarch64 cmake-3.26.5 +``` + +设置并应用cmake相关的环境变量： + +```shell +jydata@node01$ echo 'export PATH="$HOME/jydata/local/cmake-3.26.5/bin:$PATH"' >> ~/jydata/setenv.sh +jydata@node01$ source ~/jydata/setenv.sh +# 查看cmake版本 +jydata@node01$ cmake --version +# 样例输出 +cmake version 3.26.5 + +CMake suite maintained and supported by Kitware (kitware.com/cmake). +``` + +#### 3.3 Clang编译安装16.0.4版本 + +```shell +jydata@node01$ mkdir ~/jydata/local/llvm-source +jydata@node01$ tar -xvf llvm-project-16.0.4.src.tar.xz -C ~/jydata/local/llvm-source/ +jydata@node01$ mkdir ~/jydata/local/llvm-16.0.4 +jydata@node01$ cd ~/jydata/local/llvm-source/llvm-project-16.0.4.src +jydata@node01$ mkdir build && cd build +``` + +先看一下系统自带的GCC版本 + +```shell +jydata@node01$ /usr/bin/gcc --version +``` + +如果系统自带的GCC版本高于10.2.0，则执行以下命令，使用系统自带GCC编译Clang： +```shell +jydata@node01$ CC=/usr/bin/gcc CXX=/usr/bin/g++ cmake -G "Unix Makefiles" -DLLVM_ENABLE_PROJECTS="clang;openmp" -DLLVM_TARGETS_TO_BUILD="host" -DCMAKE_BUILD_TYPE="Release" -DLLVM_INCLUDE_TESTS=OFF -DCMAKE_INSTALL_PREFIX="$HOME/jydata/local/llvm-16.0.4" ../llvm +``` + +如果系统自带的GCC版本低于10.2.0，则执行以下命令，使用之前安装的GCC 10.2.0编译Clang： + +```shell +jydata@node01$ CC=$GCC_10_HOME/bin/gcc CXX=$GCC_10_HOME/bin/g++ cmake -G "Unix Makefiles" -DLLVM_ENABLE_PROJECTS="clang;openmp" -DLLVM_TARGETS_TO_BUILD="host" -DCMAKE_BUILD_TYPE="Release" -DLLVM_INCLUDE_TESTS=OFF -DCMAKE_INSTALL_PREFIX="$HOME/jydata/local/llvm-16.0.4" -DGCC_INSTALL_PREFIX=$GCC_10_HOME -DCMAKE_CXX_LINK_FLAGS="-Wl,-rpath,$GCC_10_HOME/lib64 -L$GCC_10_HOME/lib64" ../llvm +``` + +然后开始编译： + +```shell +jydata@node01$ make -j 32 # make并发数可根据实际情况调整 +jydata@node01$ make install +# 添加相关环境变量 +jydata@node01$ echo 'export LLVM_16_HOME=$HOME/jydata/local/llvm-16.0.4' >> ~/jydata/setenv.sh +jydata@node01$ echo 'export CC="$LLVM_16_HOME/bin/clang"' >> ~/jydata/setenv.sh +jydata@node01$ echo 'export CXX="$LLVM_16_HOME/bin/clang++"' >> ~/jydata/setenv.sh +jydata@node01$ echo 'export PATH="$LLVM_16_HOME/bin:$PATH"' >> ~/jydata/setenv.sh +jydata@node01$ source ~/jydata/setenv.sh +# 查看Clang版本 +jydata@node01$ clang --version +# 样例输出 +clang version 16.0.4 +Target: x86_64-unknown-linux-gnu +Thread model: posix +InstalledDir: /home/jydata/jydata/local/llvm-16.0.4/bin +``` + +最后将编译出的Clang libomp库同步到其它节点的相同路径下： + +```shell +jydata@node01$ for i in {2..6}; do ssh node0$i "mkdir -p ~/jydata/local/llvm-16.0.4/lib" && scp -r ~/jydata/local/llvm-16.0.4/lib/libomp.so node0$i:~/jydata/local/llvm-16.0.4/lib/; done +``` + +#### 3.4 下载JNA jar包 + +```shell +jydata@node01$ cd ~/jydata/third-party +jydata@node01$ wget https://repo1.maven.org/maven2/net/java/dev/jna/jna/5.6.0/jna-5.6.0.jar +jydata@node01$ mv jna-5.6.0.jar net.java.dev.jna_jna-5.6.0.jar +``` + +## TPC-DS测试环境搭建 + +### 1. TPC-DS测试所需软件部署 + +#### 1.1 HDFS及Yarn集群部署 + +在``node01`` ~ ``node06``节点部署Apache Hadoop HDFS和Yarn集群，需要Apache Hadoop 3.2及以上版本。 + +首先在``node01`` ~ ``node06``下载并安装Apache Hadoop和Hive的依赖，Java JDK 8u202。 +注意这里的JDK版本与jydata使用的JDK版本不同，原因是Hive 3.1.3不支持JDK 11版本。 + +x86_64环境： + +```shell +jydata@node01~node06$ tar -zxvf jdk-8u202-linux-x64.tar.gz -C ~/jydata/local +``` + +aarch64环境： + +```shell +jydata@node01~node06$ tar -zxvf jdk-8u202-linux-arm64-vfp-hflt.tar.gz -C ~/jydata/local +``` + +然后在``node01`` ~ ``node06``每个节点运行以下命令，安装Apache Hadoop： + +x86_64环境： + +```shell +jydata@node01~node06$ tar -zxvf hadoop-3.3.6.tar.gz -C ~/jydata/local +``` + +aarch64环境： + +```shell +jydata@node01~node06$ tar -zxvf hadoop-3.3.6-aarch64.tar.gz -C ~/jydata/local +``` + +然后在各集群节点配置HDFS和Yarn。请根据您的集群实际情况进行配置。 +``~/jydata/third-party/conf/hadoop/``下提供了配置文件样例供参考。 + +```shell +jydata@node01~node06$ cd ~/jydata/local/hadoop-3.3.6/etc/hadoop +jydata@node01~node06$ echo "export JAVA_HOME=$HOME/jydata/local/jdk1.8.0_202" >> hadoop-env.sh +jydata@node01~node06$ echo "export HADOOP_HOME=$HOME/jydata/local/hadoop-3.3.6" >> hadoop-env.sh +jydata@node01~node06$ echo "export HADOOP_CONF_DIR=$HOME/jydata/local/hadoop-3.3.6/etc/hadoop" >> hadoop-env.sh +jydata@node01~node06$ vi hadoop-env.sh # 请参考样例根据实际情况作适当配置 +jydata@node01~node06$ vi core-site.xml # 请参考样例根据实际情况作适当配置 +jydata@node01~node06$ vi hdfs-site.xml # 请参考样例根据实际情况作适当配置 +jydata@node01~node06$ vi yarn-site.xml # 请参考样例根据实际情况作适当配置 +jydata@node01~node06$ vi workers # 请参考样例根据实际情况作适当配置 +jydata@node01~node06$ vi ~/.ssh/authorized_keys # 配置SSH免密码登录，请参考样例根据实际情况作适当配置 +# 添加相关环境变量 +jydata@node01~node06$ echo 'export HADOOP_HOME="$HOME/jydata/local/hadoop-3.3.6"' >> ~/jydata/setenv.sh +jydata@node01~node06$ echo 'export HADOOP_CONF_DIR="$HADOOP_HOME/etc/hadoop"' >> ~/jydata/setenv.sh +jydata@node01~node06$ echo 'export PATH="$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH"' >> ~/jydata/setenv.sh +jydata@node01~node06$ source ~/jydata/setenv.sh +``` + +最后启动HDFS和Yarn集群： + +```shell +jydata@node01$ for i in {1..6}; do scp $SPARK_HOME/yarn/spark-3.1.2-yarn-shuffle.jar node0$i:$HADOOP_HOME/share/hadoop/yarn/lib/; done +jydata@node01$ $HADOOP_HOME/bin/hdfs namenode -format +jydata@node01$ $HADOOP_HOME/sbin/start-dfs.sh +jydata@node01$ $HADOOP_HOME/sbin/start-yarn.sh +# 检查HDFS和Yarn集群启动是否成功 +jydata@node01$ $HADOOP_HOME/bin/hadoop dfsadmin -report +jydata@node01$ $HADOOP_HOME/bin/yarn node -list -showDetails +``` + +#### 1.2 Hive部署 + +大数据智能分析系统使用Apache Hive的MetaStore作为元数据存储。 +接下来需要在``node01``节点下载并安装Hive： + +```shell +jydata@node01$ tar -zxvf apache-hive-3.1.3-bin.tar.gz -C ~/jydata/local +``` + +然后配置Hive。请根据您的集群实际情况进行配置。``~/jydata/third-party/conf/hive/``下提供了配置文件样例供参考。 + +```shell +jydata@node01$ cd ~/jydata/local/apache-hive-3.1.3-bin/conf +jydata@node01$ cp hive-env.sh.template hive-env.sh +jydata@node01$ cp hive-exec-log4j2.properties.template hive-exec-log4j2.properties +jydata@node01$ cp hive-log4j2.properties.template hive-log4j2.properties +jydata@node01$ echo "export HADOOP_HOME=$HOME/jydata/local/hadoop-3.3.6" >> hive-env.sh +jydata@node01$ echo "export HIVE_CONF_DIR=$HOME/jydata/local/apache-hive-3.1.3-bin/conf" >> hive-env.sh +jydata@node01$ $HADOOP_HOME/bin/hadoop fs -mkdir -p /user/hive/warehouse +jydata@node01$ $HADOOP_HOME/bin/hadoop fs -mkdir -p /tmp/hive +jydata@node01$ $HADOOP_HOME/bin/hadoop fs -chmod -R 777 /user/hive/warehouse +jydata@node01$ $HADOOP_HOME/bin/hadoop fs -chmod -R 777 /tmp/hive +jydata@node01$ cp ~/jydata/third-party/conf/hive/hive-site.xml ./hive-site.xml # 若有必要请根据实际情况作适当配置 +# 添加相关环境变量 +jydata@node01$ echo 'export HIVE_HOME="$HOME/jydata/local/apache-hive-3.1.3-bin"' >> ~/jydata/setenv.sh +jydata@node01$ echo 'export PATH="$HIVE_HOME/bin:$PATH"' >> ~/jydata/setenv.sh +jydata@node01$ source ~/jydata/setenv.sh +``` + +最后启动Hive的MetaStore服务。执行以下命令，验证Hive MetaStore服务可启动成功且保持运行： + +```shell +jydata@node01$ $HIVE_HOME/bin/schematool -dbType derby -initSchema +jydata@node01$ $HIVE_HOME/bin/hive --service metastore +``` + +若要使Hive MetaStore服务在后台运行，可使用nohup： + +```shell +jydata@node01$ nohup $HIVE_HOME/bin/hive --service metastore > /dev/null 2>&1 & +``` + +### 2. TPC-DS测试数据生成 + +#### 2.1 TPC-DS工具包编译 + +TPC-DS工具包包含了生成测试数据的程序，需要先对其进行编译： + +```shell +jydata@node01$ source ~/jydata/setenv.sh +jydata@node01$ git clone https://github.com/databricks/tpcds-kit.git +jydata@node01$ mv ./tpcds-kit ~/jydata/local/DSGen +jydata@node01$ cd ~/jydata/local/DSGen/tools +jydata@node01$ patch -p0 < ~/jydata/third-party/patches/tpcds_makefile.patch +jydata@node01$ make # 您可能需要自行安装GCC编译器 +``` + +最后将编译出的TPC-DS工具包同步到其它节点的相同路径下： + +```shell +jydata@node01$ for i in {2..6}; do scp -r ~/jydata/local/DSGen node0$i:~/jydata/local/; done +``` + +#### 2.2 生成TPC-DS 10T测试数据 + +```shell +jydata@node01$ source ~/jydata/setenv.sh +jydata@node01$ git clone https://github.com/databricks/spark-sql-perf.git +jydata@node01$ mv spark-sql-perf ~/jydata/local/ +jydata@node01$ cd ~/jydata/local/spark-sql-perf +jydata@node01$ sbt package # 您可能需要自行安装sbt等Scala编译器 +jydata@node01$ cd ~/jydata/sources/CMR/sbin/ +jydata@node01$ sed -i 's@$HOME@'"$HOME"'@g' tpcds_datagen_parquet.scala +jydata@node01$ $HADOOP_HOME/bin/hadoop fs -mkdir -p /user/hive/tpc-ds-10T/decimaldata +jydata@node01$ ./tpcds_datagen_parquet.sh + +# 验证TPC-DS 10T测试数据表生成成功 +jydata@node01$ $HADOOP_HOME/bin/hadoop fs -ls /user/hive/tpc-ds-10T/decimaldata +#样例输出 +Found 24 items +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/call_center +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/catalog_page +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:25 /user/hive/tpc-ds-10T/decimaldata/catalog_returns +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:23 /user/hive/tpc-ds-10T/decimaldata/catalog_sales +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/customer +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/customer_address +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/customer_demographics +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/date_dim +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/household_demographics +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/income_band +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:25 /user/hive/tpc-ds-10T/decimaldata/inventory +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/item +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/promotion +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/reason +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/ship_mode +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/store +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:30 /user/hive/tpc-ds-10T/decimaldata/store_returns +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:28 /user/hive/tpc-ds-10T/decimaldata/store_sales +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/time_dim +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/warehouse +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/web_page +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/web_returns +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:32 /user/hive/tpc-ds-10T/decimaldata/web_sales +drwxr-xr-x - jydata supergroup 0 2023-09-15 17:34 /user/hive/tpc-ds-10T/decimaldata/web_site +``` + +## TPC-DS测试运行 + +运行如下命令，开始运行TPC-DS测试： + +```shell +jydata@node01$ source ~/jydata/setenv.sh +jydata@node01$ tar -zxvf ~/jydata/third-party/tpcds_3_2.tar.gz -C ~/jydata/local/spark-sql-perf/src/main/resources +jydata@node01$ cd ~/jydata/sources/CMR/sbin/ +jydata@node01$ source ../dev/vcpkg/env.sh +jydata@node01$ ./tpcds_test.sh + +# tpcds_test.sh的核心命令是通过spark-submit提交任务。其主要参数介绍如下： +spark-submit \ + --master yarn \ #提交任务到Yarn集群 + --deploy-mode client \ + --num-executors 24 \ #executors数量 + --executor-cores 7 \ #每个executor分配的核数 + --executor-memory 30g \ #每个executor分配的内存 + --driver-memory 40g \ #driver分配的内存 + --conf ... \ #各种Spark配置项 + --jars $HOME/jydata/third-party/net.java.dev.jna_jna-5.6.0.jar \ #额外的依赖库，Java JNA + --class org.pacman.cmr.applications.LaunchSQLTpcdsPerformance \ #程序入口类，通过其中的main函数开始执行 + $HOME/jydata/sources/CMR/cmr-spark-adapter/target/scala-2.12/cmr-spark-adapter_2.12-0.5.1.jar \ #jydata Laucher的jar包 + $HOME/jydata/sources/CMR \ #jydata参数1，jydata项目源代码的根目录 + $HOME/jydata/sources/CMR/build_tpcds \ #jydata参数2，jydata项目的编译目录 + $HOME/jydata/local/spark-sql-perf/src/main/resources/tpcds_3_2 \ #jydata参数3，TPC-DS查询的SQL文件所在目录 + thrift://localhost:9083 \ #jydata参数4，Hive MetaStore服务URI + ${QID} \ #jydata参数5，运行第几个TPC-DS查询。0~102对应TPC-DS的q1.sql~q99.sql + ${NUM_REPEATS} \ #jydata参数6，重复运行次数，脚本默认3次 + ${DATABASE_NAME} \ #jydata参数7，Hive元数据库名称，在本文档中是tpcds10t + 2>&1 | tee rel_base_output_sf${SF}_query${QID}.log #运行log的保存文件 +``` + +``~/jydata/sources/CMR/sbin/tpcds_test.sh``脚本会依次执行TPC-DS的每个查询。 +每个查询分别在jydata和原生Spark中执行3次，并记录在jydata和原生Spark中的执行时间。 + +执行log保存在``~/jydata/sources/CMR/sbin/rel_base_output_sf1_queryXX.log``文件。 + +例如，对TPC-DS的q1.sql查询，查看jydata运行结果： + +```shell +jydata@node01$ grep 'cmr_ans' rel_base_output_sf1_query0.log +# 样例输出 +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 0: AAAAAAAAAAAAAABA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 1: AAAAAAAAAAAAADBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 2: AAAAAAAAAAAAAEDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 3: AAAAAAAAAAAAAOBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 4: AAAAAAAAAAAABEAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 5: AAAAAAAAAAAABNDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 6: AAAAAAAAAAAABOCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 7: AAAAAAAAAAAACCCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 8: AAAAAAAAAAAACDAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 9: AAAAAAAAAAAACEDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 10: AAAAAAAAAAAACGCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 11: AAAAAAAAAAAACLCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 12: AAAAAAAAAAAADLBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 13: AAAAAAAAAAAAEEDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 14: AAAAAAAAAAAAEHDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 15: AAAAAAAAAAAAEOAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 16: AAAAAAAAAAAAFFAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 17: AAAAAAAAAAAAFKAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 18: AAAAAAAAAAAAFMDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 19: AAAAAAAAAAAAGABA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 20: AAAAAAAAAAAAGDBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 21: AAAAAAAAAAAAGDCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 22: AAAAAAAAAAAAGKAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 23: AAAAAAAAAAAAGNBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 24: AAAAAAAAAAAAGPBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 25: AAAAAAAAAAAAHIAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 26: AAAAAAAAAAAAHIDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 27: AAAAAAAAAAAAHLAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 28: AAAAAAAAAAAAICCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 29: AAAAAAAAAAAAIHDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 30: AAAAAAAAAAAAIIAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 31: AAAAAAAAAAAAIJAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 32: AAAAAAAAAAAAIKBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 33: AAAAAAAAAAAAJIBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 34: AAAAAAAAAAAAJLAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 35: AAAAAAAAAAAAKCDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 36: AAAAAAAAAAAAKFAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 37: AAAAAAAAAAAALIBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 38: AAAAAAAAAAAALJAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 39: AAAAAAAAAAAAMGDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 40: AAAAAAAAAAAAMJDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 41: AAAAAAAAAAAANIBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 42: AAAAAAAAAAAAOEDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 43: AAAAAAAAAAAAOHCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 44: AAAAAAAAAAAAOJCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 45: AAAAAAAAAAAAOKAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 46: AAAAAAAAAAAAOOAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 47: AAAAAAAAAAAAPDDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 48: AAAAAAAAAAAAPJDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 49: AAAAAAAAAAAAPNBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 50: AAAAAAAAAAAAPNCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 51: AAAAAAAAAAAAPPBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 52: AAAAAAAAAAABACCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 53: AAAAAAAAAAABADDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 54: AAAAAAAAAAABAIBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 55: AAAAAAAAAAABAMAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 56: AAAAAAAAAAABANBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 57: AAAAAAAAAAABBCDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 58: AAAAAAAAAAABBCDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 59: AAAAAAAAAAABBFDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 60: AAAAAAAAAAABBLDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 61: AAAAAAAAAAABCDCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 62: AAAAAAAAAAABCHCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 63: AAAAAAAAAAABCHCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 64: AAAAAAAAAAABCKBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 65: AAAAAAAAAAABCMCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 66: AAAAAAAAAAABCOAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 67: AAAAAAAAAAABDCCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 68: AAAAAAAAAAABDFDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 69: AAAAAAAAAAABDLDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 70: AAAAAAAAAAABEDAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 71: AAAAAAAAAAABEFCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 72: AAAAAAAAAAABFNBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 73: AAAAAAAAAAABGJAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 74: AAAAAAAAAAABGNCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 75: AAAAAAAAAAABHIDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 76: AAAAAAAAAAABIAAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 77: AAAAAAAAAAABIGCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 78: AAAAAAAAAAABIJAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 79: AAAAAAAAAAABILBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 80: AAAAAAAAAAABJJBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 81: AAAAAAAAAAABKABA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 82: AAAAAAAAAAABKEBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 83: AAAAAAAAAAABKFAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 84: AAAAAAAAAAABKGBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 85: AAAAAAAAAAABKKCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 86: AAAAAAAAAAABKKCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 87: AAAAAAAAAAABLABA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 88: AAAAAAAAAAABLBBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 89: AAAAAAAAAAABLFDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 90: AAAAAAAAAAABLHCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 91: AAAAAAAAAAABLMDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 92: AAAAAAAAAAABLOAA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 93: AAAAAAAAAAABLOBA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 94: AAAAAAAAAAABMACA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 95: AAAAAAAAAAABMCDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 96: AAAAAAAAAAABMKCA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 97: AAAAAAAAAAABMKDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 98: AAAAAAAAAAABNBDA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 99: AAAAAAAAAAABNECA | +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1442] cmr_ans_size: 100 +[2023-09-19 09:09:39.048] [stderr] [info] [SqlOperator.h:1444] cmr_ans_display_time: 0.000261 sec +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 0: AAAAAAAAAAAAAABA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 1: AAAAAAAAAAAAADBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 2: AAAAAAAAAAAAAEDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 3: AAAAAAAAAAAAAOBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 4: AAAAAAAAAAAABEAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 5: AAAAAAAAAAAABNDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 6: AAAAAAAAAAAABOCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 7: AAAAAAAAAAAACCCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 8: AAAAAAAAAAAACDAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 9: AAAAAAAAAAAACEDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 10: AAAAAAAAAAAACGCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 11: AAAAAAAAAAAACLCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 12: AAAAAAAAAAAADLBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 13: AAAAAAAAAAAAEEDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 14: AAAAAAAAAAAAEHDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 15: AAAAAAAAAAAAEOAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 16: AAAAAAAAAAAAFFAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 17: AAAAAAAAAAAAFKAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 18: AAAAAAAAAAAAFMDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 19: AAAAAAAAAAAAGABA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 20: AAAAAAAAAAAAGDBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 21: AAAAAAAAAAAAGDCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 22: AAAAAAAAAAAAGKAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 23: AAAAAAAAAAAAGNBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 24: AAAAAAAAAAAAGPBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 25: AAAAAAAAAAAAHIAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 26: AAAAAAAAAAAAHIDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 27: AAAAAAAAAAAAHLAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 28: AAAAAAAAAAAAICCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 29: AAAAAAAAAAAAIHDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 30: AAAAAAAAAAAAIIAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 31: AAAAAAAAAAAAIJAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 32: AAAAAAAAAAAAIKBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 33: AAAAAAAAAAAAJIBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 34: AAAAAAAAAAAAJLAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 35: AAAAAAAAAAAAKCDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 36: AAAAAAAAAAAAKFAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 37: AAAAAAAAAAAALIBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 38: AAAAAAAAAAAALJAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 39: AAAAAAAAAAAAMGDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 40: AAAAAAAAAAAAMJDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 41: AAAAAAAAAAAANIBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 42: AAAAAAAAAAAAOEDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 43: AAAAAAAAAAAAOHCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 44: AAAAAAAAAAAAOJCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 45: AAAAAAAAAAAAOKAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 46: AAAAAAAAAAAAOOAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 47: AAAAAAAAAAAAPDDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 48: AAAAAAAAAAAAPJDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 49: AAAAAAAAAAAAPNBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 50: AAAAAAAAAAAAPNCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 51: AAAAAAAAAAAAPPBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 52: AAAAAAAAAAABACCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 53: AAAAAAAAAAABADDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 54: AAAAAAAAAAABAIBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 55: AAAAAAAAAAABAMAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 56: AAAAAAAAAAABANBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 57: AAAAAAAAAAABBCDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 58: AAAAAAAAAAABBCDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 59: AAAAAAAAAAABBFDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 60: AAAAAAAAAAABBLDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 61: AAAAAAAAAAABCDCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 62: AAAAAAAAAAABCHCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 63: AAAAAAAAAAABCHCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 64: AAAAAAAAAAABCKBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 65: AAAAAAAAAAABCMCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 66: AAAAAAAAAAABCOAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 67: AAAAAAAAAAABDCCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 68: AAAAAAAAAAABDFDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 69: AAAAAAAAAAABDLDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 70: AAAAAAAAAAABEDAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 71: AAAAAAAAAAABEFCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 72: AAAAAAAAAAABFNBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 73: AAAAAAAAAAABGJAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 74: AAAAAAAAAAABGNCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 75: AAAAAAAAAAABHIDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 76: AAAAAAAAAAABIAAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 77: AAAAAAAAAAABIGCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 78: AAAAAAAAAAABIJAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 79: AAAAAAAAAAABILBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 80: AAAAAAAAAAABJJBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 81: AAAAAAAAAAABKABA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 82: AAAAAAAAAAABKEBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 83: AAAAAAAAAAABKFAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 84: AAAAAAAAAAABKGBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 85: AAAAAAAAAAABKKCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 86: AAAAAAAAAAABKKCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 87: AAAAAAAAAAABLABA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 88: AAAAAAAAAAABLBBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 89: AAAAAAAAAAABLFDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 90: AAAAAAAAAAABLHCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 91: AAAAAAAAAAABLMDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 92: AAAAAAAAAAABLOAA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 93: AAAAAAAAAAABLOBA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 94: AAAAAAAAAAABMACA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 95: AAAAAAAAAAABMCDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 96: AAAAAAAAAAABMKCA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 97: AAAAAAAAAAABMKDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 98: AAAAAAAAAAABNBDA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 99: AAAAAAAAAAABNECA | +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1442] cmr_ans_size: 100 +[2023-09-19 09:10:10.990] [stderr] [info] [SqlOperator.h:1444] cmr_ans_display_time: 0.000137 sec +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 0: AAAAAAAAAAAAAABA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 1: AAAAAAAAAAAAADBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 2: AAAAAAAAAAAAAEDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 3: AAAAAAAAAAAAAOBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 4: AAAAAAAAAAAABEAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 5: AAAAAAAAAAAABNDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 6: AAAAAAAAAAAABOCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 7: AAAAAAAAAAAACCCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 8: AAAAAAAAAAAACDAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 9: AAAAAAAAAAAACEDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 10: AAAAAAAAAAAACGCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 11: AAAAAAAAAAAACLCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 12: AAAAAAAAAAAADLBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 13: AAAAAAAAAAAAEEDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 14: AAAAAAAAAAAAEHDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 15: AAAAAAAAAAAAEOAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 16: AAAAAAAAAAAAFFAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 17: AAAAAAAAAAAAFKAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 18: AAAAAAAAAAAAFMDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 19: AAAAAAAAAAAAGABA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 20: AAAAAAAAAAAAGDBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 21: AAAAAAAAAAAAGDCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 22: AAAAAAAAAAAAGKAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 23: AAAAAAAAAAAAGNBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 24: AAAAAAAAAAAAGPBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 25: AAAAAAAAAAAAHIAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 26: AAAAAAAAAAAAHIDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 27: AAAAAAAAAAAAHLAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 28: AAAAAAAAAAAAICCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 29: AAAAAAAAAAAAIHDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 30: AAAAAAAAAAAAIIAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 31: AAAAAAAAAAAAIJAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 32: AAAAAAAAAAAAIKBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 33: AAAAAAAAAAAAJIBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 34: AAAAAAAAAAAAJLAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 35: AAAAAAAAAAAAKCDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 36: AAAAAAAAAAAAKFAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 37: AAAAAAAAAAAALIBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 38: AAAAAAAAAAAALJAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 39: AAAAAAAAAAAAMGDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 40: AAAAAAAAAAAAMJDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 41: AAAAAAAAAAAANIBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 42: AAAAAAAAAAAAOEDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 43: AAAAAAAAAAAAOHCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 44: AAAAAAAAAAAAOJCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 45: AAAAAAAAAAAAOKAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 46: AAAAAAAAAAAAOOAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 47: AAAAAAAAAAAAPDDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 48: AAAAAAAAAAAAPJDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 49: AAAAAAAAAAAAPNBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 50: AAAAAAAAAAAAPNCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 51: AAAAAAAAAAAAPPBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 52: AAAAAAAAAAABACCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 53: AAAAAAAAAAABADDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 54: AAAAAAAAAAABAIBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 55: AAAAAAAAAAABAMAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 56: AAAAAAAAAAABANBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 57: AAAAAAAAAAABBCDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 58: AAAAAAAAAAABBCDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 59: AAAAAAAAAAABBFDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 60: AAAAAAAAAAABBLDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 61: AAAAAAAAAAABCDCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 62: AAAAAAAAAAABCHCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 63: AAAAAAAAAAABCHCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 64: AAAAAAAAAAABCKBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 65: AAAAAAAAAAABCMCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 66: AAAAAAAAAAABCOAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 67: AAAAAAAAAAABDCCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 68: AAAAAAAAAAABDFDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 69: AAAAAAAAAAABDLDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 70: AAAAAAAAAAABEDAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 71: AAAAAAAAAAABEFCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 72: AAAAAAAAAAABFNBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 73: AAAAAAAAAAABGJAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 74: AAAAAAAAAAABGNCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 75: AAAAAAAAAAABHIDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 76: AAAAAAAAAAABIAAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 77: AAAAAAAAAAABIGCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 78: AAAAAAAAAAABIJAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 79: AAAAAAAAAAABILBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 80: AAAAAAAAAAABJJBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 81: AAAAAAAAAAABKABA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 82: AAAAAAAAAAABKEBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 83: AAAAAAAAAAABKFAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 84: AAAAAAAAAAABKGBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 85: AAAAAAAAAAABKKCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 86: AAAAAAAAAAABKKCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 87: AAAAAAAAAAABLABA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 88: AAAAAAAAAAABLBBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 89: AAAAAAAAAAABLFDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 90: AAAAAAAAAAABLHCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 91: AAAAAAAAAAABLMDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 92: AAAAAAAAAAABLOAA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 93: AAAAAAAAAAABLOBA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 94: AAAAAAAAAAABMACA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 95: AAAAAAAAAAABMCDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 96: AAAAAAAAAAABMKCA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 97: AAAAAAAAAAABMKDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 98: AAAAAAAAAAABNBDA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1439] cmr_ans 0 99: AAAAAAAAAAABNECA | +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1442] cmr_ans_size: 100 +[2023-09-19 09:10:37.844] [stderr] [info] [SqlOperator.h:1444] cmr_ans_display_time: 7.2e-05 sec +``` + +查看原生Spark运行结果： + +```shell +jydata@node01$ grep 'spark_ans' rel_base_output_sf1_query0.log +# 样例输出 +spark_ans 0 0: AAAAAAAAAAAAAABA | +spark_ans 0 1: AAAAAAAAAAAAADBA | +spark_ans 0 2: AAAAAAAAAAAAAEDA | +spark_ans 0 3: AAAAAAAAAAAAAOBA | +spark_ans 0 4: AAAAAAAAAAAABEAA | +spark_ans 0 5: AAAAAAAAAAAABNDA | +spark_ans 0 6: AAAAAAAAAAAABOCA | +spark_ans 0 7: AAAAAAAAAAAACCCA | +spark_ans 0 8: AAAAAAAAAAAACDAA | +spark_ans 0 9: AAAAAAAAAAAACEDA | +spark_ans 0 10: AAAAAAAAAAAACGCA | +spark_ans 0 11: AAAAAAAAAAAACLCA | +spark_ans 0 12: AAAAAAAAAAAADLBA | +spark_ans 0 13: AAAAAAAAAAAAEEDA | +spark_ans 0 14: AAAAAAAAAAAAEHDA | +spark_ans 0 15: AAAAAAAAAAAAEOAA | +spark_ans 0 16: AAAAAAAAAAAAFFAA | +spark_ans 0 17: AAAAAAAAAAAAFKAA | +spark_ans 0 18: AAAAAAAAAAAAFMDA | +spark_ans 0 19: AAAAAAAAAAAAGABA | +spark_ans 0 20: AAAAAAAAAAAAGDBA | +spark_ans 0 21: AAAAAAAAAAAAGDCA | +spark_ans 0 22: AAAAAAAAAAAAGKAA | +spark_ans 0 23: AAAAAAAAAAAAGNBA | +spark_ans 0 24: AAAAAAAAAAAAGPBA | +spark_ans 0 25: AAAAAAAAAAAAHIAA | +spark_ans 0 26: AAAAAAAAAAAAHIDA | +spark_ans 0 27: AAAAAAAAAAAAHLAA | +spark_ans 0 28: AAAAAAAAAAAAICCA | +spark_ans 0 29: AAAAAAAAAAAAIHDA | +spark_ans 0 30: AAAAAAAAAAAAIIAA | +spark_ans 0 31: AAAAAAAAAAAAIJAA | +spark_ans 0 32: AAAAAAAAAAAAIKBA | +spark_ans 0 33: AAAAAAAAAAAAJIBA | +spark_ans 0 34: AAAAAAAAAAAAJLAA | +spark_ans 0 35: AAAAAAAAAAAAKCDA | +spark_ans 0 36: AAAAAAAAAAAAKFAA | +spark_ans 0 37: AAAAAAAAAAAALIBA | +spark_ans 0 38: AAAAAAAAAAAALJAA | +spark_ans 0 39: AAAAAAAAAAAAMGDA | +spark_ans 0 40: AAAAAAAAAAAAMJDA | +spark_ans 0 41: AAAAAAAAAAAANIBA | +spark_ans 0 42: AAAAAAAAAAAAOEDA | +spark_ans 0 43: AAAAAAAAAAAAOHCA | +spark_ans 0 44: AAAAAAAAAAAAOJCA | +spark_ans 0 45: AAAAAAAAAAAAOKAA | +spark_ans 0 46: AAAAAAAAAAAAOOAA | +spark_ans 0 47: AAAAAAAAAAAAPDDA | +spark_ans 0 48: AAAAAAAAAAAAPJDA | +spark_ans 0 49: AAAAAAAAAAAAPNBA | +spark_ans 0 50: AAAAAAAAAAAAPNCA | +spark_ans 0 51: AAAAAAAAAAAAPPBA | +spark_ans 0 52: AAAAAAAAAAABACCA | +spark_ans 0 53: AAAAAAAAAAABADDA | +spark_ans 0 54: AAAAAAAAAAABAIBA | +spark_ans 0 55: AAAAAAAAAAABAMAA | +spark_ans 0 56: AAAAAAAAAAABANBA | +spark_ans 0 57: AAAAAAAAAAABBCDA | +spark_ans 0 58: AAAAAAAAAAABBCDA | +spark_ans 0 59: AAAAAAAAAAABBFDA | +spark_ans 0 60: AAAAAAAAAAABBLDA | +spark_ans 0 61: AAAAAAAAAAABCDCA | +spark_ans 0 62: AAAAAAAAAAABCHCA | +spark_ans 0 63: AAAAAAAAAAABCHCA | +spark_ans 0 64: AAAAAAAAAAABCKBA | +spark_ans 0 65: AAAAAAAAAAABCMCA | +spark_ans 0 66: AAAAAAAAAAABCOAA | +spark_ans 0 67: AAAAAAAAAAABDCCA | +spark_ans 0 68: AAAAAAAAAAABDFDA | +spark_ans 0 69: AAAAAAAAAAABDLDA | +spark_ans 0 70: AAAAAAAAAAABEDAA | +spark_ans 0 71: AAAAAAAAAAABEFCA | +spark_ans 0 72: AAAAAAAAAAABFNBA | +spark_ans 0 73: AAAAAAAAAAABGJAA | +spark_ans 0 74: AAAAAAAAAAABGNCA | +spark_ans 0 75: AAAAAAAAAAABHIDA | +spark_ans 0 76: AAAAAAAAAAABIAAA | +spark_ans 0 77: AAAAAAAAAAABIGCA | +spark_ans 0 78: AAAAAAAAAAABIJAA | +spark_ans 0 79: AAAAAAAAAAABILBA | +spark_ans 0 80: AAAAAAAAAAABJJBA | +spark_ans 0 81: AAAAAAAAAAABKABA | +spark_ans 0 82: AAAAAAAAAAABKEBA | +spark_ans 0 83: AAAAAAAAAAABKFAA | +spark_ans 0 84: AAAAAAAAAAABKGBA | +spark_ans 0 85: AAAAAAAAAAABKKCA | +spark_ans 0 86: AAAAAAAAAAABKKCA | +spark_ans 0 87: AAAAAAAAAAABLABA | +spark_ans 0 88: AAAAAAAAAAABLBBA | +spark_ans 0 89: AAAAAAAAAAABLFDA | +spark_ans 0 90: AAAAAAAAAAABLHCA | +spark_ans 0 91: AAAAAAAAAAABLMDA | +spark_ans 0 92: AAAAAAAAAAABLOAA | +spark_ans 0 93: AAAAAAAAAAABLOBA | +spark_ans 0 94: AAAAAAAAAAABMACA | +spark_ans 0 95: AAAAAAAAAAABMCDA | +spark_ans 0 96: AAAAAAAAAAABMKCA | +spark_ans 0 97: AAAAAAAAAAABMKDA | +spark_ans 0 98: AAAAAAAAAAABNBDA | +spark_ans 0 99: AAAAAAAAAAABNECA | +spark_ans_size: 100 [correct] +spark_ans 0 0: AAAAAAAAAAAAAABA | +spark_ans 0 1: AAAAAAAAAAAAADBA | +spark_ans 0 2: AAAAAAAAAAAAAEDA | +spark_ans 0 3: AAAAAAAAAAAAAOBA | +spark_ans 0 4: AAAAAAAAAAAABEAA | +spark_ans 0 5: AAAAAAAAAAAABNDA | +spark_ans 0 6: AAAAAAAAAAAABOCA | +spark_ans 0 7: AAAAAAAAAAAACCCA | +spark_ans 0 8: AAAAAAAAAAAACDAA | +spark_ans 0 9: AAAAAAAAAAAACEDA | +spark_ans 0 10: AAAAAAAAAAAACGCA | +spark_ans 0 11: AAAAAAAAAAAACLCA | +spark_ans 0 12: AAAAAAAAAAAADLBA | +spark_ans 0 13: AAAAAAAAAAAAEEDA | +spark_ans 0 14: AAAAAAAAAAAAEHDA | +spark_ans 0 15: AAAAAAAAAAAAEOAA | +spark_ans 0 16: AAAAAAAAAAAAFFAA | +spark_ans 0 17: AAAAAAAAAAAAFKAA | +spark_ans 0 18: AAAAAAAAAAAAFMDA | +spark_ans 0 19: AAAAAAAAAAAAGABA | +spark_ans 0 20: AAAAAAAAAAAAGDBA | +spark_ans 0 21: AAAAAAAAAAAAGDCA | +spark_ans 0 22: AAAAAAAAAAAAGKAA | +spark_ans 0 23: AAAAAAAAAAAAGNBA | +spark_ans 0 24: AAAAAAAAAAAAGPBA | +spark_ans 0 25: AAAAAAAAAAAAHIAA | +spark_ans 0 26: AAAAAAAAAAAAHIDA | +spark_ans 0 27: AAAAAAAAAAAAHLAA | +spark_ans 0 28: AAAAAAAAAAAAICCA | +spark_ans 0 29: AAAAAAAAAAAAIHDA | +spark_ans 0 30: AAAAAAAAAAAAIIAA | +spark_ans 0 31: AAAAAAAAAAAAIJAA | +spark_ans 0 32: AAAAAAAAAAAAIKBA | +spark_ans 0 33: AAAAAAAAAAAAJIBA | +spark_ans 0 34: AAAAAAAAAAAAJLAA | +spark_ans 0 35: AAAAAAAAAAAAKCDA | +spark_ans 0 36: AAAAAAAAAAAAKFAA | +spark_ans 0 37: AAAAAAAAAAAALIBA | +spark_ans 0 38: AAAAAAAAAAAALJAA | +spark_ans 0 39: AAAAAAAAAAAAMGDA | +spark_ans 0 40: AAAAAAAAAAAAMJDA | +spark_ans 0 41: AAAAAAAAAAAANIBA | +spark_ans 0 42: AAAAAAAAAAAAOEDA | +spark_ans 0 43: AAAAAAAAAAAAOHCA | +spark_ans 0 44: AAAAAAAAAAAAOJCA | +spark_ans 0 45: AAAAAAAAAAAAOKAA | +spark_ans 0 46: AAAAAAAAAAAAOOAA | +spark_ans 0 47: AAAAAAAAAAAAPDDA | +spark_ans 0 48: AAAAAAAAAAAAPJDA | +spark_ans 0 49: AAAAAAAAAAAAPNBA | +spark_ans 0 50: AAAAAAAAAAAAPNCA | +spark_ans 0 51: AAAAAAAAAAAAPPBA | +spark_ans 0 52: AAAAAAAAAAABACCA | +spark_ans 0 53: AAAAAAAAAAABADDA | +spark_ans 0 54: AAAAAAAAAAABAIBA | +spark_ans 0 55: AAAAAAAAAAABAMAA | +spark_ans 0 56: AAAAAAAAAAABANBA | +spark_ans 0 57: AAAAAAAAAAABBCDA | +spark_ans 0 58: AAAAAAAAAAABBCDA | +spark_ans 0 59: AAAAAAAAAAABBFDA | +spark_ans 0 60: AAAAAAAAAAABBLDA | +spark_ans 0 61: AAAAAAAAAAABCDCA | +spark_ans 0 62: AAAAAAAAAAABCHCA | +spark_ans 0 63: AAAAAAAAAAABCHCA | +spark_ans 0 64: AAAAAAAAAAABCKBA | +spark_ans 0 65: AAAAAAAAAAABCMCA | +spark_ans 0 66: AAAAAAAAAAABCOAA | +spark_ans 0 67: AAAAAAAAAAABDCCA | +spark_ans 0 68: AAAAAAAAAAABDFDA | +spark_ans 0 69: AAAAAAAAAAABDLDA | +spark_ans 0 70: AAAAAAAAAAABEDAA | +spark_ans 0 71: AAAAAAAAAAABEFCA | +spark_ans 0 72: AAAAAAAAAAABFNBA | +spark_ans 0 73: AAAAAAAAAAABGJAA | +spark_ans 0 74: AAAAAAAAAAABGNCA | +spark_ans 0 75: AAAAAAAAAAABHIDA | +spark_ans 0 76: AAAAAAAAAAABIAAA | +spark_ans 0 77: AAAAAAAAAAABIGCA | +spark_ans 0 78: AAAAAAAAAAABIJAA | +spark_ans 0 79: AAAAAAAAAAABILBA | +spark_ans 0 80: AAAAAAAAAAABJJBA | +spark_ans 0 81: AAAAAAAAAAABKABA | +spark_ans 0 82: AAAAAAAAAAABKEBA | +spark_ans 0 83: AAAAAAAAAAABKFAA | +spark_ans 0 84: AAAAAAAAAAABKGBA | +spark_ans 0 85: AAAAAAAAAAABKKCA | +spark_ans 0 86: AAAAAAAAAAABKKCA | +spark_ans 0 87: AAAAAAAAAAABLABA | +spark_ans 0 88: AAAAAAAAAAABLBBA | +spark_ans 0 89: AAAAAAAAAAABLFDA | +spark_ans 0 90: AAAAAAAAAAABLHCA | +spark_ans 0 91: AAAAAAAAAAABLMDA | +spark_ans 0 92: AAAAAAAAAAABLOAA | +spark_ans 0 93: AAAAAAAAAAABLOBA | +spark_ans 0 94: AAAAAAAAAAABMACA | +spark_ans 0 95: AAAAAAAAAAABMCDA | +spark_ans 0 96: AAAAAAAAAAABMKCA | +spark_ans 0 97: AAAAAAAAAAABMKDA | +spark_ans 0 98: AAAAAAAAAAABNBDA | +spark_ans 0 99: AAAAAAAAAAABNECA | +spark_ans_size: 100 [correct] +spark_ans 0 0: AAAAAAAAAAAAAABA | +spark_ans 0 1: AAAAAAAAAAAAADBA | +spark_ans 0 2: AAAAAAAAAAAAAEDA | +spark_ans 0 3: AAAAAAAAAAAAAOBA | +spark_ans 0 4: AAAAAAAAAAAABEAA | +spark_ans 0 5: AAAAAAAAAAAABNDA | +spark_ans 0 6: AAAAAAAAAAAABOCA | +spark_ans 0 7: AAAAAAAAAAAACCCA | +spark_ans 0 8: AAAAAAAAAAAACDAA | +spark_ans 0 9: AAAAAAAAAAAACEDA | +spark_ans 0 10: AAAAAAAAAAAACGCA | +spark_ans 0 11: AAAAAAAAAAAACLCA | +spark_ans 0 12: AAAAAAAAAAAADLBA | +spark_ans 0 13: AAAAAAAAAAAAEEDA | +spark_ans 0 14: AAAAAAAAAAAAEHDA | +spark_ans 0 15: AAAAAAAAAAAAEOAA | +spark_ans 0 16: AAAAAAAAAAAAFFAA | +spark_ans 0 17: AAAAAAAAAAAAFKAA | +spark_ans 0 18: AAAAAAAAAAAAFMDA | +spark_ans 0 19: AAAAAAAAAAAAGABA | +spark_ans 0 20: AAAAAAAAAAAAGDBA | +spark_ans 0 21: AAAAAAAAAAAAGDCA | +spark_ans 0 22: AAAAAAAAAAAAGKAA | +spark_ans 0 23: AAAAAAAAAAAAGNBA | +spark_ans 0 24: AAAAAAAAAAAAGPBA | +spark_ans 0 25: AAAAAAAAAAAAHIAA | +spark_ans 0 26: AAAAAAAAAAAAHIDA | +spark_ans 0 27: AAAAAAAAAAAAHLAA | +spark_ans 0 28: AAAAAAAAAAAAICCA | +spark_ans 0 29: AAAAAAAAAAAAIHDA | +spark_ans 0 30: AAAAAAAAAAAAIIAA | +spark_ans 0 31: AAAAAAAAAAAAIJAA | +spark_ans 0 32: AAAAAAAAAAAAIKBA | +spark_ans 0 33: AAAAAAAAAAAAJIBA | +spark_ans 0 34: AAAAAAAAAAAAJLAA | +spark_ans 0 35: AAAAAAAAAAAAKCDA | +spark_ans 0 36: AAAAAAAAAAAAKFAA | +spark_ans 0 37: AAAAAAAAAAAALIBA | +spark_ans 0 38: AAAAAAAAAAAALJAA | +spark_ans 0 39: AAAAAAAAAAAAMGDA | +spark_ans 0 40: AAAAAAAAAAAAMJDA | +spark_ans 0 41: AAAAAAAAAAAANIBA | +spark_ans 0 42: AAAAAAAAAAAAOEDA | +spark_ans 0 43: AAAAAAAAAAAAOHCA | +spark_ans 0 44: AAAAAAAAAAAAOJCA | +spark_ans 0 45: AAAAAAAAAAAAOKAA | +spark_ans 0 46: AAAAAAAAAAAAOOAA | +spark_ans 0 47: AAAAAAAAAAAAPDDA | +spark_ans 0 48: AAAAAAAAAAAAPJDA | +spark_ans 0 49: AAAAAAAAAAAAPNBA | +spark_ans 0 50: AAAAAAAAAAAAPNCA | +spark_ans 0 51: AAAAAAAAAAAAPPBA | +spark_ans 0 52: AAAAAAAAAAABACCA | +spark_ans 0 53: AAAAAAAAAAABADDA | +spark_ans 0 54: AAAAAAAAAAABAIBA | +spark_ans 0 55: AAAAAAAAAAABAMAA | +spark_ans 0 56: AAAAAAAAAAABANBA | +spark_ans 0 57: AAAAAAAAAAABBCDA | +spark_ans 0 58: AAAAAAAAAAABBCDA | +spark_ans 0 59: AAAAAAAAAAABBFDA | +spark_ans 0 60: AAAAAAAAAAABBLDA | +spark_ans 0 61: AAAAAAAAAAABCDCA | +spark_ans 0 62: AAAAAAAAAAABCHCA | +spark_ans 0 63: AAAAAAAAAAABCHCA | +spark_ans 0 64: AAAAAAAAAAABCKBA | +spark_ans 0 65: AAAAAAAAAAABCMCA | +spark_ans 0 66: AAAAAAAAAAABCOAA | +spark_ans 0 67: AAAAAAAAAAABDCCA | +spark_ans 0 68: AAAAAAAAAAABDFDA | +spark_ans 0 69: AAAAAAAAAAABDLDA | +spark_ans 0 70: AAAAAAAAAAABEDAA | +spark_ans 0 71: AAAAAAAAAAABEFCA | +spark_ans 0 72: AAAAAAAAAAABFNBA | +spark_ans 0 73: AAAAAAAAAAABGJAA | +spark_ans 0 74: AAAAAAAAAAABGNCA | +spark_ans 0 75: AAAAAAAAAAABHIDA | +spark_ans 0 76: AAAAAAAAAAABIAAA | +spark_ans 0 77: AAAAAAAAAAABIGCA | +spark_ans 0 78: AAAAAAAAAAABIJAA | +spark_ans 0 79: AAAAAAAAAAABILBA | +spark_ans 0 80: AAAAAAAAAAABJJBA | +spark_ans 0 81: AAAAAAAAAAABKABA | +spark_ans 0 82: AAAAAAAAAAABKEBA | +spark_ans 0 83: AAAAAAAAAAABKFAA | +spark_ans 0 84: AAAAAAAAAAABKGBA | +spark_ans 0 85: AAAAAAAAAAABKKCA | +spark_ans 0 86: AAAAAAAAAAABKKCA | +spark_ans 0 87: AAAAAAAAAAABLABA | +spark_ans 0 88: AAAAAAAAAAABLBBA | +spark_ans 0 89: AAAAAAAAAAABLFDA | +spark_ans 0 90: AAAAAAAAAAABLHCA | +spark_ans 0 91: AAAAAAAAAAABLMDA | +spark_ans 0 92: AAAAAAAAAAABLOAA | +spark_ans 0 93: AAAAAAAAAAABLOBA | +spark_ans 0 94: AAAAAAAAAAABMACA | +spark_ans 0 95: AAAAAAAAAAABMCDA | +spark_ans 0 96: AAAAAAAAAAABMKCA | +spark_ans 0 97: AAAAAAAAAAABMKDA | +spark_ans 0 98: AAAAAAAAAAABNBDA | +spark_ans 0 99: AAAAAAAAAAABNECA | +spark_ans_size: 100 [correct] +``` + +jydata与Spark的运行结果一致，证明jydata运行正确。 + +查看jydata运行时间： + +```shell +jydata@node01$ grep 'cmr_time' rel_base_output_sf1_query0.log +# 样例输出 +cmr_time 0 10.472 sec +cmr_time 1 5.4510000000000005 sec +cmr_time 2 5.309 sec +``` + +查看原生Spark运行时间： + +```shell +jydata@node01$ grep 'spark_time' rel_base_output_sf1_query0.log +# 样例输出 +spark_time 0 16.096 sec ( + 1.932 sec for analysis) +spark_time 1 11.684000000000001 sec ( + 1.359 sec for analysis) +spark_time 2 10.949 sec ( + 1.424 sec for analysis) +``` + +取后两次运行时间的平均值作为查询的运行时间，则jydata对q1.sql查询的运行时间为5.38s，原生Spark的运行时间为11.3165s。 diff --git a/README.md b/README.md new file mode 100644 index 0000000..cc4c3ef --- /dev/null +++ b/README.md @@ -0,0 +1,30 @@ +# 九源大数据智能分析系统 + +九源大数据智能分析系统jydata在Apache Spark的基础上构建了一套以性能为中心的高效数据处理系统。 +一方面实现了一套基于C++的RDD编程接口和高性能大数据算子库以支持高性能大数据应用的编写； +另一方面提出了动静态结合的计算图使得在容错能力上等同Apache Spark。 +同时，在数据预处理、图计算、机器学习等领域典型应用上进行了测试，取得了多个指标的性能优势。 +该系统目前已支持TPC-DS中所有99个查询，且在相同环境下相比Apache Spark SQL达到了3.0倍以上的加速比。 +该系统可以降低大数据处理的整体延迟、并可节省大数据处理的成本； +同时能够无缝部署在现有Apache Spark环境中，部署简单。因此，大数据智能分析系统具有显著的应用前景。 +大数据智能分析系统支持x64和aarch64架构，已在国产鲲鹏和飞腾服务器完成验证。 + +![jydata](https://osredm.com/repo/jiuyuan/jydata/raw/branch/master/images/jydata.png "jydata") + +jydataSQL通过在C++上实现DataFrame编程接口，融合jydata过程式编程框架与原生SQL执行引擎。jydataSQL的原生SQL执行引擎采用代码生成器替换的方法，将SparkSQL的代码生成器替换成jydataSQL的代码生成器。具体而言，SparkSQL的代码生成器面向的是Tungsten执行引擎，而jydataSQL的代码生成器面向的是jydata。这种构建原生SQL执行引擎的方式可以在对齐SparkSQL的查询能力的基础上原生加速SparkSQL。 + +![jydataSQL_workflow](https://osredm.com/repo/jiuyuan/jydata/raw/branch/master/images/jydataSQL_workflow.png "jydataSQL_workflow") + +jydataSQL分成两个模块，分别是编程接口与SQL执行引擎。 + +![jydataSQL_architecture](https://osredm.com/repo/jiuyuan/jydata/raw/branch/master/images/jydataSQL_architecture.png "jydataSQL_architecture") + +jydataSQL提供原生的、基于C++实现的DataFrame编程接口，整合了jydata引擎提供的过程式编程能力与SQL执行引擎提供的关系型数据分析能力，用户可以根据负载特性在通用但难编程的过程式与易编程但有限的关系型之间自由选择。jydataSQL同时还兼容JDBC/ODBC与控制台接口，该接口并非jydataSQL实现，而是由于复用了SparkSQL的Catalyst优化器从而复用了它的编程接口。 + +jydataSQL还提供原生的SQL执行引擎。在原有的Spark模块架构中，SparkSQL通过Catalyst查询优化器，将多种编程接口所编写的程序翻译成物理计划。物理计划在通过SparkSQL代码生成器进行算子融合后，执行在Tungsten引擎上。而jydataSQL将原有的SparkSQL代码生成器替换成jydataSQL代码生成器，将物理计划整体变换成C++代码后交由jydata引擎执行。jydata引擎包含用C++实现的高性能算子库，并提供低开销的Spark整合。 + +这种架构设计在利用原生编译获得高性能特性的同时，保留了SparkSQL的优化器以达成功能复用，并保留了Spark核心以支持大数据需要的高可扩展、低开销容错。这使得jydataSQL不仅在性能方面较SparkSQL有优势，在功能方面还能与SparkSQL对齐。 + +## 部署文档 + +[部署文档](https://www.osredm.com/jiuyuan/jydata/tree/master/ProjectDeploy.md "部署文档") \ No newline at end of file diff --git a/images/jydata.png b/images/jydata.png new file mode 100644 index 0000000..1a95f9a Binary files /dev/null and b/images/jydata.png differ diff --git a/images/jydataSQL_architecture.png b/images/jydataSQL_architecture.png new file mode 100644 index 0000000..b807463 Binary files /dev/null and b/images/jydataSQL_architecture.png differ diff --git a/images/jydataSQL_workflow.png b/images/jydataSQL_workflow.png new file mode 100644 index 0000000..7e4259b Binary files /dev/null and b/images/jydataSQL_workflow.png differ diff --git a/jydata_release_2024_04_10_aarch64.tar.gz b/jydata_release_2024_04_10_aarch64.tar.gz new file mode 100644 index 0000000..709a6ad Binary files /dev/null and b/jydata_release_2024_04_10_aarch64.tar.gz differ diff --git a/jydata_release_2024_04_10_x64.tar.gz b/jydata_release_2024_04_10_x64.tar.gz new file mode 100644 index 0000000..6e23889 Binary files /dev/null and b/jydata_release_2024_04_10_x64.tar.gz differ