Skip to content

Commit

Permalink
Deployed 62dae23 with MkDocs version: 1.5.3
Browse files Browse the repository at this point in the history
  • Loading branch information
@yanbinyang
yanbinyang committed Aug 26, 2024
1 parent 64d4e94 commit 39375ff
Show file tree
Hide file tree
Showing 21 changed files with 278 additions and 206 deletions.
2 changes: 1 addition & 1 deletion index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -528,5 +528,5 @@ <h4 class="modal-title" id="keyboardModalLabel">Keyboard Shortcuts</h4>

<!--
MkDocs version : 1.5.3
Build Date UTC : 2024-08-22 01:56:52.630064+00:00
Build Date UTC : 2024-08-26 00:33:46.342472+00:00
-->
2 changes: 1 addition & 1 deletion jindosdk/hadoop/jindosdk_ide_hadoop/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,7 @@ <h1 id="hadoop-jindosdk-ide">Hadoop 使用 JindoSDK 在 IDE 开发调试<a class
&lt;version&gt;1.0&lt;/version&gt;

&lt;properties&gt;
&lt;jindodata.version&gt;6.5.4&lt;/jindodata.version&gt;
&lt;jindodata.version&gt;6.6.0&lt;/jindodata.version&gt;
&lt;hadoop.version&gt;2.8.5&lt;/hadoop.version&gt;
&lt;/properties&gt;

Expand Down
6 changes: 3 additions & 3 deletions jindosdk/hbase/jindosdk_on_hbase/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -423,9 +423,9 @@ <h2 id="2-jar">2. 安装 jar 包<a class="headerlink" href="#2-jar" title="Perma
<p>解压下载的安装包，将安装包内的以下 jar 文件安装到 hadoop 的 classpath 下：
* jindo-core-x.x.x.jar
* jindo-sdk-x.x.x.jar</p>
<p>jindosdk-6.5.4 为例:</p>
<pre><code>cp jindosdk-6.5.4/lib/jindo-core-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.5.4/lib/jindo-sdk-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
<p>jindosdk-6.6.0 为例:</p>
<pre><code>cp jindosdk-6.6.0/lib/jindo-core-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.6.0/lib/jindo-sdk-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
</code></pre>
<h2 id="3-oss-hdfs-access-key">3. 配置 OSS-HDFS 服务实现类及 Access Key<a class="headerlink" href="#3-oss-hdfs-access-key" title="Permanent link">§</a></h2>
<p>将 JindoSDK OSS 实现类配置到 Hadoop 的<code>core-site.xml</code>中。</p>
Expand Down
74 changes: 65 additions & 9 deletions jindosdk/jindosdk_configuration/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -465,36 +465,36 @@ <h2 id="ossoss-hdfs">OSS/OSS-HDFS<a class="headerlink" href="#ossoss-hdfs" title
<td>fs.oss.upload.thread.concurrency</td>
<td>整型</td>
<td>MAX(cpu核数,16)</td>
<td>单个文件 OSS/OSS-HDFS 并发上传线程数</td>
<td>4.3.0+</td>
<td>进程内 OSS/OSS-HDFS 并发上传线程数</td>
<td>4.3.0+(deprecated)</td>
</tr>
<tr>
<td>fs.oss.upload.queue.size</td>
<td>整型</td>
<td>MAX(cpu核数,16)</td>
<td>OSS/OSS-HDFS 并发上传任务队列大小</td>
<td>4.3.0+</td>
<td>进程内 OSS/OSS-HDFS 并发上传任务队列大小</td>
<td>4.3.0+(deprecated)</td>
</tr>
<tr>
<td>fs.oss.upload.max.pending.tasks.per.stream</td>
<td>整型</td>
<td>10</td>
<td>进程内 oss 最大并发上传任务数</td>
<td>4.3.0+</td>
<td>单个文件 OSS/OSS-HDFS 最大并发上传任务数</td>
<td>4.3.0+(deprecated)</td>
</tr>
<tr>
<td>fs.oss.download.thread.concurrency</td>
<td>整型</td>
<td>MAX(cpu核数,16)</td>
<td>进程内 oss 最大并发下载任务数</td>
<td>4.3.0+</td>
<td>进程内 OSS/OSS-HDFS 最大并发下载任务数</td>
<td>4.3.0+(deprecated)</td>
</tr>
<tr>
<td>fs.oss.read.readahead.max.buffer.count</td>
<td>整型</td>
<td>48</td>
<td>最大同时预读 oss 的 buffer 个数</td>
<td>4.5.1+</td>
<td>4.5.1+(deprecated)</td>
</tr>
<tr>
<td>fs.oss.read.buffer.size</td>
Expand Down Expand Up @@ -573,6 +573,62 @@ <h2 id="ossoss-hdfs">OSS/OSS-HDFS<a class="headerlink" href="#ossoss-hdfs" title
<td>签名算法版本，可选值[0，1，4]。推荐使用0，表示默认算法版本，6.3.0版本之后默认使用 V4 签名算法提高访问 OSS/OSS-HDFS，若特殊场景希望指定签名算法版本，请指定 1 或 4。</td>
<td>6.3.0+</td>
</tr>
<tr>
<td>fs.oss.io.timeout.millisecond</td>
<td>整型</td>
<td>MAX(cpu核数,16)</td>
<td>请求 OSS/OSS-HDFS 超时时间（毫秒）</td>
<td>6.6.0 nextarch+</td>
</tr>
<tr>
<td>fs.oss.async.async.executor.number</td>
<td>整型</td>
<td>cpu核数</td>
<td>进程内 OSS/OSS-HDFS 异步io线程数</td>
<td>6.6.0 nextarch+</td>
</tr>
<tr>
<td>fs.oss.upload.async.concurrency</td>
<td>整型</td>
<td>MAX(cpu核数,16)</td>
<td>进程内 OSS/OSS-HDFS 异步上传并发数</td>
<td>6.6.0 nextarch+</td>
</tr>
<tr>
<td>fs.oss.download.async.concurrency</td>
<td>整型</td>
<td>MAX(cpu核数,16)</td>
<td>进程内 OSS/OSS-HDFS 异步下载并发数</td>
<td>6.6.0 nextarch+</td>
</tr>
<tr>
<td>fs.jdo.metrics.metrics.level</td>
<td>整型</td>
<td>1</td>
<td>mertrics 收集等级，0是不收集，1是静态metrics，2是包含 bucket 信息的动态 metrics，3包含所有 metrics</td>
<td>6.6.0 nextarch+</td>
</tr>
<tr>
<td>fs.jdo.metrics.file.enable</td>
<td>整型</td>
<td>false</td>
<td>控制是否输出 metrics 到文件</td>
<td>6.6.0 nextarch+</td>
</tr>
<tr>
<td>fs.jdo.metrics.file.dir</td>
<td>字符串</td>
<td>/tmp/metrics</td>
<td>输出 metrics 到文件的所在目录</td>
<td>6.6.0 nextarch+</td>
</tr>
<tr>
<td>fs.jdo.metrics.interval.sec</td>
<td>整型</td>
<td>15</td>
<td>输出 metrics 到文件的间隔时间</td>
<td>6.6.0 nextarch+</td>
</tr>
</tbody>
</table>
<p>Credential Provider 相关配置，详见<a href="../jindosdk_credential_provider/">《配置 OSS/OSS-HDFS Credential Provider》</a></p></div>
Expand Down
10 changes: 5 additions & 5 deletions jindosdk/jindosdk_deployment_ai/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -409,19 +409,19 @@ <h2 id="jindosdk">部署 JindoSDK<a class="headerlink" href="#jindosdk" title="P
<ol>
<li>下载并解压 JindoSDK TAR 包。</li>
</ol>
<p>执行以下命令，下载6.5.4版本JindoSDK TAR包。以大多数linux x86环境为例。其他平台部署参见<a href="../jindosdk_deployment_multi_platform/">《在多平台环境安装部署 JindoSDK》</a></p>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/6.5.4/jindosdk-6.5.4-linux.tar.gz
<p>执行以下命令，下载6.6.0版本JindoSDK TAR包。以大多数linux x86环境为例。其他平台部署参见<a href="../jindosdk_deployment_multi_platform/">《在多平台环境安装部署 JindoSDK》</a></p>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/6.6.0/jindosdk-6.6.0-linux.tar.gz
</code></pre>
<ol>
<li>执行以下命令，解压 JindoSDK TAR 包。</li>
</ol>
<pre><code class="language-bash">tar zxvf jindosdk-6.5.4-linux.tar.gz
<pre><code class="language-bash">tar zxvf jindosdk-6.6.0-linux.tar.gz
</code></pre>
<ol>
<li>配置环境变量。</li>
</ol>
<p>以安装包内容解压在 /usr/lib/jindosdk-6.5.4-linux 目录为例：</p>
<pre><code class="language-bash">export JINDOSDK_HOME=/usr/lib/jindosdk-6.5.4-linux
<p>以安装包内容解压在 /usr/lib/jindosdk-6.6.0-linux 目录为例：</p>
<pre><code class="language-bash">export JINDOSDK_HOME=/usr/lib/jindosdk-6.6.0-linux
export JINDOSDK_CONF_DIR=${JINDOSDK_HOME}/conf
export PATH=${PATH}:${JINDOSDK_HOME}/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JINDOSDK_HOME}/lib/native
Expand Down
10 changes: 5 additions & 5 deletions jindosdk/jindosdk_deployment_hadoop/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -408,19 +408,19 @@ <h2 id="jindosdk">部署 JindoSDK<a class="headerlink" href="#jindosdk" title="P
<ol>
<li>下载并解压 JindoSDK TAR 包。</li>
</ol>
<p>执行以下命令，下载6.5.4版本JindoSDK TAR包。以大多数linux x86环境为例。其他平台部署参见<a href="../jindosdk_deployment_multi_platform/">《在多平台环境安装部署 JindoSDK》</a></p>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/6.5.4/jindosdk-6.5.4-linux.tar.gz
<p>执行以下命令，下载6.6.0版本JindoSDK TAR包。以大多数linux x86环境为例。其他平台部署参见<a href="../jindosdk_deployment_multi_platform/">《在多平台环境安装部署 JindoSDK》</a></p>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/6.6.0/jindosdk-6.6.0-linux.tar.gz
</code></pre>
<ol>
<li>执行以下命令，解压 JindoSDK TAR 包。</li>
</ol>
<pre><code class="language-bash">tar zxvf jindosdk-6.5.4-linux.tar.gz
<pre><code class="language-bash">tar zxvf jindosdk-6.6.0-linux.tar.gz
</code></pre>
<ol>
<li>配置环境变量。</li>
</ol>
<p>以安装包内容解压在 /usr/lib/jindosdk-6.5.4-linux 目录为例：</p>
<pre><code class="language-bash">export JINDOSDK_HOME=/usr/lib/jindosdk-6.5.4-linux
<p>以安装包内容解压在 /usr/lib/jindosdk-6.6.0-linux 目录为例：</p>
<pre><code class="language-bash">export JINDOSDK_HOME=/usr/lib/jindosdk-6.6.0-linux
export JINDOSDK_CONF_DIR=${JINDOSDK_HOME}/conf
export PATH=${PATH}:${JINDOSDK_HOME}/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JINDOSDK_HOME}/lib/native
Expand Down
12 changes: 6 additions & 6 deletions jindosdk/jindosdk_deployment_lite_hadoop/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -398,19 +398,19 @@ <h2 id="jindosdk">部署 JindoSDK<a class="headerlink" href="#jindosdk" title="P
<ol>
<li>下载 JindoSDK JAR 包。</li>
</ol>
<p>执行以下命令，下载6.5.4版本JindoSDK JAR包。以大多数linux x86环境为例。其他平台部署参见<a href="../jindosdk_deployment_multi_platform/">《在多平台环境安装部署 JindoSDK》</a></p>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-sdk/6.5.4/jindo-sdk-6.5.4.jar
wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-core/6.5.4/jindo-core-6.5.4.jar
<p>执行以下命令，下载6.6.0版本JindoSDK JAR包。以大多数linux x86环境为例。其他平台部署参见<a href="../jindosdk_deployment_multi_platform/">《在多平台环境安装部署 JindoSDK》</a></p>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-sdk/6.6.0/jindo-sdk-6.6.0.jar
wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-core/6.6.0/jindo-core-6.6.0.jar
</code></pre>
<ol>
<li>安装 JindoSDK JAR 包
将下载的 jar 文件安装到 hadoop 的 classpath 下：</li>
<li>jindo-core-x.x.x.jar</li>
<li>jindo-sdk-x.x.x.jar</li>
</ol>
<p>以 jindosdk-6.5.4 为例:</p>
<pre><code>cp jindo-core-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
<p>以 jindosdk-6.6.0 为例:</p>
<pre><code>cp jindo-core-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
</code></pre>
<p><strong>重要</strong> 请将安装目录和环境变量部署到所有所需节点上。</p>
<h2 id="hadoop">Hadoop 配置文件<a class="headerlink" href="#hadoop" title="Permanent link">§</a></h2>
Expand Down
40 changes: 20 additions & 20 deletions jindosdk/jindosdk_deployment_multi_platform/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -428,7 +428,7 @@ <h3 id="_2">完整产出物说明<a class="headerlink" href="#_2" title="Permane
<h3 id="lite">lite 产出物<a class="headerlink" href="#lite" title="Permanent link">§</a></h3>
<p>如果希望在 Hadoop 环境中仅使用 JindoSDK 中的 hadoop sdk，可以使用此类产出物。</p>
<p>使用Java SDK时，Linux X86平台需要依赖 <code>jindo-sdk-x.y.z.jar</code> 和 <code>jindo-core-x.y.z.jar</code>，其他平台需要依赖 <code>jindo-sdk-x.y.z.jar</code>、<code>jindo-core-x.y.z.jar</code> 和 <code>jindo-core-&lt;平台名称&gt;-x.y.z</code>.jar。</p>
<p>安装方式的话，即可以通过maven在pom.xml中配置依赖，如<a href="../oss-maven/">6.5.4-oss-maven</a>。也可以通过oss地址直接下载部署，如<a href="../jindosdk_download/">6.5.4-download</a>。</p>
<p>安装方式的话，即可以通过maven在pom.xml中配置依赖，如<a href="../oss-maven/">6.6.0-oss-maven</a>。也可以通过oss地址直接下载部署，如<a href="../jindosdk_download/">6.6.0-download</a>。</p>
<h3 id="_3">产物物平台说明表<a class="headerlink" href="#_3" title="Permanent link">§</a></h3>
<table>
<thead>
Expand Down Expand Up @@ -492,16 +492,16 @@ <h3 id="_3">产物物平台说明表<a class="headerlink" href="#_3" title="Perm
</tr>
</tbody>
</table>
<p>ps: 6.5.4 版本开始 tar 包中的 lite jar 默认使用 nextarch classifer，即 jindo-core-x.y.z-nextarch.jar、jindo-core-linux-el7-aarch64-x.y.z-nextarch.jar 。。</p>
<p>ps: 6.6.0 版本开始 tar 包中的 lite jar 默认使用 nextarch classifer，即 jindo-core-x.y.z-nextarch.jar、jindo-core-linux-el7-aarch64-x.y.z-nextarch.jar 。。</p>
<h2 id="jindosdk_2">JindoSDK 部署示例<a class="headerlink" href="#jindosdk_2" title="Permanent link">§</a></h2>
<p>以下 6.5.4 版本为例。</p>
<p>以下 6.6.0 版本为例。</p>
<h3 id="linux-x86">Linux x86 环境示例<a class="headerlink" href="#linux-x86" title="Permanent link">§</a></h3>
<ol>
<li>完整安装，以安装包内容解压在 /usr/lib/jindosdk-6.5.4-linux 目录为例：</li>
<li>完整安装，以安装包内容解压在 /usr/lib/jindosdk-6.6.0-linux 目录为例：</li>
</ol>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/6.5.4/jindosdk-6.5.4-linux.tar.gz
tar zxvf jindosdk-6.5.4-linux.tar.gz -C /usr/lib/
export JINDOSDK_HOME=/usr/lib/jindosdk-6.5.4-linux
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/6.6.0/jindosdk-6.6.0-linux.tar.gz
tar zxvf jindosdk-6.6.0-linux.tar.gz -C /usr/lib/
export JINDOSDK_HOME=/usr/lib/jindosdk-6.6.0-linux
export JINDOSDK_CONF_DIR=${JINDOSDK_HOME}/conf
export PATH=${PATH}:${JINDOSDK_HOME}/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JINDOSDK_HOME}/lib/native
Expand All @@ -510,18 +510,18 @@ <h3 id="linux-x86">Linux x86 环境示例<a class="headerlink" href="#linux-x86"
<ol>
<li>lite安装，以安装包内容下载到 <HADOOP_HOME>/share/hadoop/hdfs/lib/ 目录为例：</li>
</ol>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-sdk/6.5.4/jindo-sdk-6.5.4.jar
wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-core/6.5.4/jindo-core-6.5.4.jar
cp jindo-core-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-sdk/6.6.0/jindo-sdk-6.6.0.jar
wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-core/6.6.0/jindo-core-6.6.0.jar
cp jindo-core-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
</code></pre>
<h3 id="linux-aarch64">Linux aarch64 环境示例<a class="headerlink" href="#linux-aarch64" title="Permanent link">§</a></h3>
<ol>
<li>完整安装，以安装包内容解压在 /usr/lib/jindosdk-6.5.4-linux-el7-aarch64 目录为例：</li>
<li>完整安装，以安装包内容解压在 /usr/lib/jindosdk-6.6.0-linux-el7-aarch64 目录为例：</li>
</ol>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/x.y.z/jindosdk-6.5.4-linux-el7-aarch64.tar.gz
tar zxvf jindosdk-6.5.4-linux-el7-aarch64.tar.gz -C /usr/lib/
export JINDOSDK_HOME=/usr/lib/jindosdk-6.5.4-linux-el7-aarch64
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/release/x.y.z/jindosdk-6.6.0-linux-el7-aarch64.tar.gz
tar zxvf jindosdk-6.6.0-linux-el7-aarch64.tar.gz -C /usr/lib/
export JINDOSDK_HOME=/usr/lib/jindosdk-6.6.0-linux-el7-aarch64
export JINDOSDK_CONF_DIR=${JINDOSDK_HOME}/conf
export PATH=${PATH}:${JINDOSDK_HOME}/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JINDOSDK_HOME}/lib/native
Expand All @@ -530,11 +530,11 @@ <h3 id="linux-aarch64">Linux aarch64 环境示例<a class="headerlink" href="#li
<ol>
<li>lite安装，以安装包内容下载到 <HADOOP_HOME>/share/hadoop/hdfs/lib/ 目录为例：</li>
</ol>
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-sdk/6.5.4/jindo-sdk-6.5.4.jar
wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-core-linux-el7-aarch64/6.5.4/jindo-core-linux-el7-aarch64-6.5.4.jar
cp jindo-core-linux-el7-aarch64-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindo-core-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.5.4.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
<pre><code class="language-bash">wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-sdk/6.6.0/jindo-sdk-6.6.0.jar
wget https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/com/aliyun/jindodata/jindo-core-linux-el7-aarch64/6.6.0/jindo-core-linux-el7-aarch64-6.6.0.jar
cp jindo-core-linux-el7-aarch64-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindo-core-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
cp jindosdk-6.6.0.jar &lt;HADOOP_HOME&gt;/share/hadoop/hdfs/lib/
</code></pre>
<h1 id="_4">附录<a class="headerlink" href="#_4" title="Permanent link">§</a></h1>
<h2 id="_5">完整产出物示例<a class="headerlink" href="#_5" title="Permanent link">§</a></h2>
Expand Down
Loading

0 comments on commit 39375ff

Please sign in to comment.