写在前面
以下操作语句按顺序执行即可，注意切换目录的命令一定记得执行，如果需要改动的地方会有${}注释，其余不需要任何改动，默认安装版本为1.12.4（稳定版）
本地化部署
下载
# 创建目录
mkdir /data/soft/doris-1.2.4
cd /data/soft/doris-1.2.4
# 下载FE、BE、解压、更名
wget https://archive.apache.org/dist/doris/1.2/1.2.4.1-rc01/apache-doris-fe-1.2.4.1-bin-x86_64.tar.xz
wget https://archive.apache.org/dist/doris/1.2/1.2.4.1-rc01/apache-doris-be-1.2.4.1-bin-x86_64.tar.xz
tar -xvf apache-doris-fe-1.2.4.1-bin-x86_64.tar.xz
tar -xvf apache-doris-be-1.2.4.1-bin-x86_64.tar.xz
mv apache-doris-fe-1.2.4.1-bin-x86_64 fe
mv apache-doris-be-1.2.4.1-bin-x86_64 be
修改FE配置文件
# 修改FE配置文件
vi /data/soft/doris-1.2.4/fe/conf/fe.conf启动FE
# 启动后检查 http://${feip}:8030,正常访问代表安装成功，否则去掉--daemon 查看日志
/data/soft/doris-1.2.4/fe/bin/start-fe.sh --daemon修改BE配置文件
# 修改BE配置文件
vi /data/soft/doris-1.2.4/be/conf/be.confBE UDF依赖
# 下载udf依赖
cd /data/soft/doris-1.2.4/fe/conf/lib
wget https://archive.apache.org/dist/doris/1.2/1.2.4.1-rc01/apache-doris-dependencies-1.2.4.1-bin-x86_64.tar.xz
# 解压
tar -xvf apache-doris-dependencies-1.2.4.1-bin-x86_64.tar.xz
# 替换文件
mv apache-doris-dependencies-1.2.4.1-bin-x86_64/* ./启动BE
/data/soft/doris-1.2.4/fe/bin/start-be.sh --daemon注册BE
# 进入一台有mysql的服务器，端口9093，账号root，默认密码是空
mysql -h ${fe的ip} -P9030 -uroot
# 注册be
ALTER SYSTEM ADD BACKEND "${be的ip}:9050";
# 查看be Alive属性如果是true就ok了
show PROC '/backends';MySQL [(none)]> show PROC '/backends';
+-----------+-----------------+---------------+----------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| BackendId | Cluster | IP | HostName | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole |
+-----------+-----------------+---------------+----------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| 10003 | default_cluster | 172.16.10.142 | hdp04 | 9050 | 9060 | 8041 | 8060 | 2023-06-15 08:42:32 | 2023-06-15 08:42:45 | true | false | false | 0 | 0.000 | 713.640 GB | 899.561 GB | 20.67 % | 20.67 % | 0.000 | {"location" : "default"} | | doris-1.2.4-1-Unknown | {"lastSuccessReportTabletsTime":"2023-06-15 08:42:39","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
+-----------+-----------------+---------------+----------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+##################以下是Docker部署################
镜像编译
准备工作
1、确认/opt/目录是否还有空间，至少需要10G，这个只是编译镜像的临时文件，会自动删除的，所以就安装在这个目录下，如果修改了，以下涉及opt的路径都要同步更改，非常不建议
df -h /opt2、切换到root用户
sudo su root3、创建文件夹
mkdir -p ~/docker-build/be/resource
mkdir -p ~/docker-build/fe/resource下载编译好的包
fe安装包
cd ~/docker-build/fe/resource
# 下载fe编译包
wget https://archive.apache.org/dist/doris/1.2/1.2.4.1-rc01/apache-doris-fe-1.2.4.1-bin-x86_64.tar.xz
# 解压xz
xz -d apache-doris-fe-1.2.4.1-bin-x86_64.tar.xz
# 压缩为gz，最后会产出一个 apache-doris-fe-1.2.4.1-bin-x86_64.tar.gz
gizp apache-doris-fe-1.2.4.1-bin-x86_64.tar
be安装包
cd ~/docker-build/be/resource
# 下载be编译包
wget https://archive.apache.org/dist/doris/1.2/1.2.4.1-rc01/apache-doris-be-1.2.4.1-bin-x86_64.tar.xz
# 解压xz
xz -d apache-doris-be-1.2.4.1-bin-x86_64.tar.xz
# 压缩为gz，最后会产出一个 apache-doris-be-1.2.4.1-bin-x86_64.tar.gz
gizp apache-doris-be-1.2.4.1-bin-x86_64.tar
编辑初始化文件
init_fe.sh
cd ~/docker-build/fe/resource
# 输入以下文件内容
vi init_fe.shinit_fe.sh内容，不需要任何改动
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.set -eo pipefail
shopt -s nullglobDORIS_HOME="/opt/apache-doris"# Obtain necessary and basic information to complete initialization# logging functions
# usage: doris_[note|warn|error] $log_meg
# ie: doris_warn "task may fe risky!"
# out: 2023-01-08T19:08:16+08:00 [Warn] [Entrypoint]: task may fe risky!
doris_log() {local type="$1"shift# accept argument string or stdinlocal text="$*"if [ "$#" -eq 0 ]; then text="$(cat)"; filocal dt="$(date -Iseconds)"printf '%s [%s] [Entrypoint]: %s\n' "$dt" "$type" "$text"
}
doris_note() {doris_log Note "$@"
}
doris_warn() {doris_log Warn "$@" >&2
}
doris_error() {doris_log ERROR "$@" >&2exit 1
}# check to see if this file is being run or sourced from another script
_is_sourced() {[ "${#FUNCNAME[@]}" -ge 2 ] &&[ "${FUNCNAME[0]}" = '_is_sourced' ] &&[ "${FUNCNAME[1]}" = 'source' ]
}docker_setup_env() {declare -g DATABASE_ALREADY_EXISTS BUILD_TYPE_K8Sif [ -d "${DORIS_HOME}/fe/doris-meta/image" ]; thenDATABASE_ALREADY_EXISTS='true'fi
}# Check the variables required for startup
docker_required_variables_env() {declare -g RUN_TYPEif [ -n "$BUILD_TYPE" ]; thenRUN_TYPE="K8S"if [[ $BUILD_TYPE =~ ^([kK]8[sS])$ ]]; thendoris_warn "BUILD_TYPE" $BUILD_TYPEelsedoris_error "BUILD_TYPE rule error！example: [k8s], Default Value: docker."fireturnfiif [[ -n "$FE_SERVERS" && -n "$FE_ID" ]]; thenRUN_TYPE="ELECTION"if [[ $FE_SERVERS =~ ^.+:[1-2]{0,1}[0-9]{0,1}[0-9]{1}(\.[1-2]{0,1}[0-9]{0,1}[0-9]{1}){3}:[1-6]{0,1}[0-9]{1,4}(,.+:[1-2]{0,1}[0-9]{0,1}[0-9]{1}(\.[1-2]{0,1}[0-9]{0,1}[0-9]{1}){3}:[1-6]{0,1}[0-9]{1,4})*$ || $FE_SERVERS =~ ^([0-9a-fA-F]{1,4}:){7,7}([0-9a-fA-F]{1,4}|:)|([0-9a-fA-F]{1,4}:){1,6}(:[0-9a-fA-F]{1,4}|:)|([0-9a-fA-F]{1,4}:){1,5}((:[0-9a-fA-F]{1,4}){1,2}|:)|([0-9a-fA-F]{1,4}:){1,4}((:[0-9a-fA-F]{1,4}){1,3}|:)|([0-9a-fA-F]{1,4}:){1,3}((:[0-9a-fA-F]{1,4}){1,4}|:)|([0-9a-fA-F]{1,4}:){1,2}((:[0-9a-fA-F]{1,4}){1,5}|:)|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6}|:)|:((:[0-9a-fA-F]{1,4}){1,7}|:)$ ]]; thendoris_warn "FE_SERVERS" $FE_SERVERSelsedoris_error "FE_SERVERS rule error！example: \$FE_NAME:\$FE_HOST_IP:\$FE_EDIT_LOG_PORT[,\$FE_NAME:\$FE_HOST_IP:\$FE_EDIT_LOG_PORT]..."fiif [[ $FE_ID =~ ^[1-9]{1}$ ]]; thendoris_warn "FE_ID" $FE_IDelsedoris_error "FE_ID rule error！If FE is the role of Master, please set FE_ID=1, and ensure that all IDs correspond to the IP of the current node."fireturnfidoris_note $FE_MASTER_IP " " $FE_MASTER_PORT " " $FE_CURRENT_IP " " $FE_CURRENT_PORTif [[ -n "$FE_MASTER_IP" && -n "$FE_MASTER_PORT" && -n "$FE_CURRENT_IP" && -n "$FE_CURRENT_PORT" ]]; thenRUN_TYPE="ASSIGN"if [[ $FE_MASTER_IP =~ ^[1-2]{0,1}[0-9]{0,1}[0-9]{1}(\.[1-2]{0,1}[0-9]{0,1}[0-9]{1}){3}$ || $FE_MASTER_IP =~ ^([0-9a-fA-F]{1,4}:){7,7}([0-9a-fA-F]{1,4}|:)|([0-9a-fA-F]{1,4}:){1,6}(:[0-9a-fA-F]{1,4}|:)|([0-9a-fA-F]{1,4}:){1,5}((:[0-9a-fA-F]{1,4}){1,2}|:)|([0-9a-fA-F]{1,4}:){1,4}((:[0-9a-fA-F]{1,4}){1,3}|:)|([0-9a-fA-F]{1,4}:){1,3}((:[0-9a-fA-F]{1,4}){1,4}|:)|([0-9a-fA-F]{1,4}:){1,2}((:[0-9a-fA-F]{1,4}){1,5}|:)|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6}|:)|:((:[0-9a-fA-F]{1,4}){1,7}|:)$ ]]; thendoris_warn "FE_MASTER_IP" $FE_MASTER_IPelsedoris_error "FE_MASTER_IP rule error！example: \$FE_MASTER_IP"fiif [[ $FE_MASTER_PORT =~ ^[1-6]{0,1}[0-9]{1,4}$ ]]; thendoris_warn "FE_MASTER_PORT" $FE_MASTER_PORTelsedoris_error "FE_MASTER_PORT rule error！example: \$FE_MASTER_EDIT_LOG_HOST."fiif [[ $FE_CURRENT_IP =~ ^[1-2]{0,1}[0-9]{0,1}[0-9]{1}(\.[1-2]{0,1}[0-9]{0,1}[0-9]{1}){3}$ || $FE_CURRENT_IP =~ ^([0-9a-fA-F]{1,4}:){7,7}([0-9a-fA-F]{1,4}|:)|([0-9a-fA-F]{1,4}:){1,6}(:[0-9a-fA-F]{1,4}|:)|([0-9a-fA-F]{1,4}:){1,5}((:[0-9a-fA-F]{1,4}){1,2}|:)|([0-9a-fA-F]{1,4}:){1,4}((:[0-9a-fA-F]{1,4}){1,3}|:)|([0-9a-fA-F]{1,4}:){1,3}((:[0-9a-fA-F]{1,4}){1,4}|:)|([0-9a-fA-F]{1,4}:){1,2}((:[0-9a-fA-F]{1,4}){1,5}|:)|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6}|:)|:((:[0-9a-fA-F]{1,4}){1,7}|:)$ ]]; thendoris_warn "FE_CURRENT_IP" $FE_CURRENT_IPelsedoris_error "FE_CURRENT_IP rule error！example: \$FE_CURRENT_IP"fiif [[ $FE_CURRENT_PORT =~ ^[1-6]{0,1}[0-9]{1,4}$ ]]; thendoris_warn "FE_CURRENT_PORT" $FE_CURRENT_PORTelsedoris_error "FE_CURRENT_PORT rule error！example: \$FE_CURRENT_EDIT_LOG_HOST."fireturnfidoris_error EOF "Note that you did not configure the required parameters!plan 1:BUILD_TYPEplan 2:FE_SERVERS & FE_IDplan 3:FE_MASTER_IP & FE_MASTER_PORT & FE_CURRENT_IP & FE_CURRENT_PORT"EOF}get_doris_fe_args() {declare -g MASTER_FE_IP CURRENT_FE_IP MASTER_FE_EDIT_PORT CURRENT_FE_EDIT_PORT PRIORITY_NETWORKS CURRENT_FE_IS_MASTERif [ $RUN_TYPE == "ELECTION" ]; thenlocal feServerArray=($(echo "${FE_SERVERS}" | awk '{gsub (/,/," "); print $0}'))for i in "${feServerArray[@]}"; doval=${i}val=${val// /}tmpFeName=$(echo "${val}" | awk -F ':' '{ sub(/fe/, ""); sub(/ /, ""); print$1}')tmpFeIp=$(echo "${val}" | awk -F ':' '{ sub(/ /, ""); print$2}')tmpFeEditLogPort=$(echo "${val}" | awk -F ':' '{ sub(/ /, ""); print$3}')check_arg "TMP_FE_NAME" $tmpFeNamefeIpArray[$tmpFeName]=${tmpFeIp}check_arg "TMP_FE_EDIT_LOG_PORT" $tmpFeEditLogPortfeEditLogPortArray[$tmpFeName]=${tmpFeEditLogPort}doneMASTER_FE_IP=${feIpArray[1]}check_arg "MASTER_FE_IP" $MASTER_FE_IPMASTER_FE_EDIT_PORT=${feEditLogPortArray[1]}check_arg "MASTER_FE_EDIT_PORT" $MASTER_FE_EDIT_PORTCURRENT_FE_IP=${feIpArray[FE_ID]}check_arg "CURRENT_FE_IP" $CURRENT_FE_IPCURRENT_FE_EDIT_PORT=${feEditLogPortArray[FE_ID]}check_arg "CURRENT_FE_EDIT_PORT" $CURRENT_FE_EDIT_PORTif [ ${MASTER_FE_IP} == ${CURRENT_FE_IP} ]; thenCURRENT_FE_IS_MASTER=trueelseCURRENT_FE_IS_MASTER=falsefiPRIORITY_NETWORKS=$(echo "${CURRENT_FE_IP}" | awk -F '.' '{print$1"."$2"."$3".0/24"}')check_arg "PRIORITY_NETWORKS" $PRIORITY_NETWORKSdoris_note "FE_IP_ARRAY = ${feIpArray[*]}"doris_note "FE_EDIT_LOG_PORT_ARRAY = ${feEditLogPortArray[*]}"doris_note "MASTER_FE = ${feIpArray[1]}:${feEditLogPortArray[1]}"doris_note "CURRENT_FE = ${CURRENT_FE_IP}:${CURRENT_FE_EDIT_PORT}"doris_note "PRIORITY_NETWORKS = ${PRIORITY_NETWORKS}"elif [ $RUN_TYPE == "ASSIGN" ]; thenMASTER_FE_IP=${FE_MASTER_IP}check_arg "MASTER_FE_IP" $MASTER_FE_IPMASTER_FE_EDIT_PORT=${FE_MASTER_PORT}check_arg "MASTER_FE_EDIT_PORT" $MASTER_FE_EDIT_PORTCURRENT_FE_IP=${FE_CURRENT_IP}check_arg "CURRENT_FE_IP" $CURRENT_FE_IPCURRENT_FE_EDIT_PORT=${FE_CURRENT_PORT}check_arg "CURRENT_FE_EDIT_PORT" $CURRENT_FE_EDIT_PORTif [ ${MASTER_FE_IP} == ${CURRENT_FE_IP} ]; thenCURRENT_FE_IS_MASTER=trueelseCURRENT_FE_IS_MASTER=falsefiPRIORITY_NETWORKS=$(echo "${CURRENT_FE_IP}" | awk -F '.' '{print$1"."$2"."$3".0/24"}')check_arg "PRIORITY_NETWORKS" $PRIORITY_NETWORKSfi# check fe startcheck_fe_status true}add_priority_networks() {doris_note "add priority_networks ${1} to ${DORIS_HOME}/fe/conf/fe.conf"echo "priority_networks = ${1}" >>${DORIS_HOME}/fe/conf/fe.conf
}# Execute sql script, passed via stdin
# usage: docker_process_sql sql_script
docker_process_sql() {set +emysql -uroot -P9030 -h${MASTER_FE_IP} --comments "$@" 2>/dev/null
}docker_setup_db() {set +e# check fe statuslocal is_fe_start=falseif [ ${CURRENT_FE_IS_MASTER} == true ]; thendoris_note "Current FE is Master FE! No need to register again！"returnfifor i in {1..300}; dodocker_process_sql <<<"alter system add FOLLOWER '${CURRENT_FE_IP}:${CURRENT_FE_EDIT_PORT}'"register_fe_status=$?if [[ $register_fe_status == 0 ]]; thendoris_note "FE successfully registered！"is_fe_start=truebreakelsecheck_fe_statusif [ -n "$CURRENT_FE_ALREADY_EXISTS" ]; thendoris_warn "Same frontend already exists! No need to register again！"breakfiif [[ $(($i % 20)) == 1 ]]; thendoris_warn "register_fe_status: ${register_fe_status}"doris_warn "FE failed registered!"fifiif [[ $(($i % 20)) == 1 ]]; thendoris_note "ADD FOLLOWER failed, retry."fisleep 1doneif ! [[ $is_fe_start ]]; thendoris_error "Failed to register CURRENT_FE to FE！Tried 30 times！Maybe FE Start Failed！"fi
}# Check whether the passed parameters are empty to avoid subsequent task execution failures. At the same time,
# enumeration checks can fe added, such as checking whether a certain parameter appears repeatedly, etc.
check_arg() {if [ -z $2 ]; thendoris_error "$1 is null!"fi
}check_fe_status() {set +edeclare -g CURRENT_FE_ALREADY_EXISTSif [[ ${CURRENT_FE_IS_MASTER} == true ]]; thendoris_note "Current FE is Master FE! No need check fe status！"returnfifor i in {1..300}; doif [[ $1 == true ]]; thendocker_process_sql <<<"show frontends" | grep "[[:space:]]${MASTER_FE_IP}[[:space:]]" | grep "[[:space:]]${MASTER_FE_EDIT_PORT}[[:space:]]"elsedocker_process_sql <<<"show frontends" | grep "[[:space:]]${CURRENT_FE_IP}[[:space:]]" | grep "[[:space:]]${CURRENT_FE_EDIT_PORT}[[:space:]]"fife_join_status=$?if [[ "${fe_join_status}" == 0 ]]; thenif [[ $1 == true ]]; thendoris_note "Master FE is started!"elsedoris_note "Verify that CURRENT_FE is registered to FE successfully"fiCURRENT_FE_ALREADY_EXISTS=truebreakelseif [[ $(($i % 20)) == 1 ]]; thenif [[ $1 == true ]]; thendoris_note "Master FE is not started, retry."elsedoris_warn "Verify that CURRENT_FE is registered to FE failed, retry."fififiif [[ $(($i % 20)) == 1 ]]; thendoris_note "try session Master FE."fisleep 1done
}cleanup() {doris_note "Container stopped, running stop_fe script"${DORIS_HOME}/fe/bin/stop_fe.sh
}_main() {docker_required_variables_envtrap 'cleanup' SIGTERM SIGINTif [[ $RUN_TYPE == "K8S" ]]; thenstart_fe.sh &child_pid=$!elsedocker_setup_envget_doris_fe_argsif [ -z "$DATABASE_ALREADY_EXISTS" ]; thenadd_priority_networks $PRIORITY_NETWORKSfidocker_setup_dbcheck_fe_statusdoris_note "Ready to start CURRENT_FE！"if [ $CURRENT_FE_IS_MASTER == true ]; thenstart_fe.sh &child_pid=$!elsestart_fe.sh --helper ${MASTER_FE_IP}:${MASTER_FE_EDIT_PORT} &child_pid=$!fifiwait $child_pidexec "$@"
}if ! _is_sourced; then_main "$@"
fiinit_be.sh
cd ~/docker-build/be/resource
# 输入以下内容
vi init_be.shinit_be.sh内容，不需要任何改动
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.set -eo pipefail
shopt -s nullglobDORIS_HOME="/opt/apache-doris"# Obtain necessary and basic information to complete initialization# logging functions
# usage: doris_[note|warn|error] $log_meg
# ie: doris_warn "task may be risky!"
# out: 2023-01-08T19:08:16+08:00 [Warn] [Entrypoint]: task may be risky!
doris_log() {local type="$1"shift# accept argument string or stdinlocal text="$*"if [ "$#" -eq 0 ]; then text="$(cat)"; filocal dt="$(date -Iseconds)"printf '%s [%s] [Entrypoint]: %s\n' "$dt" "$type" "$text"
}
doris_note() {doris_log Note "$@"
}
doris_warn() {doris_log Warn "$@" >&2
}
doris_error() {doris_log ERROR "$@" >&2exit 1
}# check to see if this file is being run or sourced from another script
_is_sourced() {[ "${#FUNCNAME[@]}" -ge 2 ] &&[ "${FUNCNAME[0]}" = '_is_sourced' ] &&[ "${FUNCNAME[1]}" = 'source' ]
}docker_setup_env() {declare -g DATABASE_ALREADY_EXISTSif [ -d "${DORIS_HOME}/be/storage/data" ]; thenDATABASE_ALREADY_EXISTS='true'fi
}add_priority_networks() {doris_note "add priority_networks ${1} to ${DORIS_HOME}/be/conf/be.conf"echo "priority_networks = ${1}" >>${DORIS_HOME}/be/conf/be.conf
}show_be_args(){doris_note "============= init args ================"doris_note "MASTER_FE_IP " ${MASTER_FE_IP}doris_note "CURRENT_BE_IP " ${CURRENT_BE_IP}doris_note "CURRENT_BE_PORT " ${CURRENT_BE_PORT}doris_note "RUN_TYPE " ${RUN_TYPE}doris_note "PRIORITY_NETWORKS " ${PRIORITY_NETWORKS}
}# Execute sql script, passed via stdin
# usage: docker_process_sql sql_script
docker_process_sql() {set +emysql -uroot -P9030 -h${MASTER_FE_IP} --comments "$@" 2>/dev/null
}node_role_conf(){if [[ ${NODE_ROLE} == 'computation' ]]; thendoris_note "this node role is computation"echo "be_node_role=computation" >>${DORIS_HOME}/be/conf/be.confelsedoris_note "this node role is mix"fi
}register_be_to_fe() {set +e# check fe statuslocal is_fe_start=falsefor i in {1..300}; dodocker_process_sql <<<"alter system add backend '${CURRENT_BE_IP}:${CURRENT_BE_PORT}'"register_be_status=$?if [[ $register_be_status == 0 ]]; thendoris_note "BE successfully registered to FE！"is_fe_start=truebreakelsecheck_be_statusif [ -n "$BE_ALREADY_EXISTS" ]; thendoris_warn "Same backend already exists! No need to register again！"breakfiif [[ $(( $i % 20 )) == 1 ]]; thendoris_warn "register_be_status: ${register_be_status}"doris_warn "BE failed registered to FE!"fifiif [[ $(( $i % 20 )) == 1 ]]; thendoris_note "Register BE to FE is failed. retry."fisleep 1doneif ! [[ $is_fe_start ]]; thendoris_error "Failed to register BE to FE！Tried 30 times！Maybe FE Start Failed！"fi
}check_be_status() {set +elocal is_fe_start=falsefor i in {1..300}; doif [[ $(($i % 20)) == 1 ]]; thendoris_warn "start check be register status~"fidocker_process_sql <<<"show backends;" | grep "[[:space:]]${CURRENT_BE_IP}[[:space:]]" | grep "[[:space:]]${CURRENT_BE_PORT}[[:space:]]"be_join_status=$?if [[ "${be_join_status}" == 0 ]]; thendoris_note "Verify that BE is registered to FE successfully"is_fe_start=truereturnelseif [[ $(($i % 20)) == 1 ]]; thendoris_note "register is failed, wait next~"fifisleep 1doneif [[ ! $is_fe_start ]]; thendoris_error "Failed to register BE to FE！Tried 30 times！Maybe FE Start Failed！"fi
}cleanup() {doris_note "Container stopped, running stop_be script"${DORIS_HOME}/be/bin/stop_be.sh
}_main() {trap 'cleanup' SIGTERM SIGINTif [[ $RUN_TYPE == "K8S" ]]; thenstart_be.sh &child_pid=$!elsedocker_setup_envif [ -z "$DATABASE_ALREADY_EXISTS" ]; thenadd_priority_networks $PRIORITY_NETWORKSnode_role_confshow_be_argsregister_be_to_feficheck_be_statusdoris_note "Ready to start BE！"start_be.sh &child_pid=$!fiwait $child_pidexec "$@"
}if ! _is_sourced; then_main "$@"
fi编辑Dockerfile
fe-Dockerfile
cd ~/docker-build/fe/
# 输入以下内容
vi DockerfileDockerfile(fe)内容，不需要任何改动
FROM openjdk:8u342-jdk# 设置环境变量
ENV JAVA_HOME="/usr/local/openjdk-8/" \PATH="/opt/apache-doris/fe/bin:$PATH"# 下载软件至镜像内，可根据需要替换
ADD ./resource/apache-doris-fe-1.2.4.1-bin-x86_64.tar.gz /opt/RUN apt-get update && \apt-get install -y default-mysql-client && \apt-get clean && \mkdir /opt/apache-doris && \cd /opt && \mv apache-doris-fe-1.2.4.1-bin-x86_64 /opt/apache-doris/feADD ./resource/init_fe.sh /opt/apache-doris/fe/bin
RUN chmod 755 /opt/apache-doris/fe/bin/init_fe.shENTRYPOINT ["/opt/apache-doris/fe/bin/init_fe.sh"]be-Dockerfile
cd ~/docker-build/be/
# 输入以下内容
vi DockerfileDockerfile(be)内容，不需要任何改动
# 选择基础镜像
FROM openjdk:8u342-jdk# 设置环境变量
ENV JAVA_HOME="/usr/local/openjdk-8/" \PATH="/opt/apache-doris/be/bin:$PATH"# 下载软件至镜像内，可根据需要替换
ADD ./resource/apache-doris-be-1.2.4.1-bin-x86_64.tar.gz /opt/RUN apt-get update && \apt-get install -y default-mysql-client && \apt-get clean && \mkdir /opt/apache-doris && \cd /opt && \mv apache-doris-be-1.2.4.1-bin-x86_64 /opt/apache-doris/beADD ./resource/init_be.sh /opt/apache-doris/be/bin
RUN chmod 755 /opt/apache-doris/be/bin/init_be.shENTRYPOINT ["/opt/apache-doris/be/bin/init_be.sh"]构建镜像
doris-fe
cd ~/docker-builder/fe
docker build . -t apache-doris:1.2.4-fe doris-be
cd ~/docker-builder/be
docker build . -t apache-doris:1.2.4-be 构建结果
[root@hdp04 be]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
apache-doris 1.2.4-be 0d8aadf01bf0 6 hours ago 7.76GB
apache-doris 1.2.4-fe 2f5961488ce7 6 hours ago 1.5GB
镜像运行
doris-fe
docker run -itd \
--name=doris-fe \
--env FE_SERVERS="fe1:${服务器ip}:9010" \
--env FE_ID=1 \
-p 8030:8030 \
-p 9030:9030 \
-v /data/docker/doris/fe/log:/opt/apache-doris/fe/log \
--net=host \
apache-doris:1.2.4-fedoris-be
注意：如果你要重新run请先删除 /data/docker/doris/be/storage
docker run -itd \
--name=doris-be \
--env FE_SERVERS="fe1:${服务器ip}:9010" \
--env BE_ADDR="${服务器ip}:9050" \
-p 8040:8040 \
-v /data/docker/doris/be/storage:/opt/apache-doris/be/storage \
-v /data/docker/doris/be/log:/opt/apache-doris/be/log \
--net=host \
apache-doris:1.2.4-be
修改Be配置
[root@hdp04 be]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f198db1d315c apache-doris:1.2.4-be "/data/soft/apache-d…" 51 minutes ago Up 51 minutes doris-be
2b7970de5fff apache-doris:1.2.4-fe "/data/soft/apache-d…" About an hour ago Up About an hour doris-fe# 进入fe容器
docker exec -it doris-fe /bin/bash
# 查看fe的网段配置priority_networks，并复制出来 (priority_networks = 172.16.10.0/24)
cat /opt/apache-doris/fe/conf/fe.conf
# 退出fe容器
exit;# 进入be容器
docker exec -it doris-fe /bin/bash
# 下载vim
apt-get install vim
vi /opt/apache-doris/be/conf/be.conf
# 修改fe的网段配置（与fe一致）
priority_networks = 172.16.10.0/24
添加udf依赖
# 进入be容器
docker exec -it doris-fe /bin/bash
# 进入lib目录
cd /opt/apache-doris/be/lib
# 下载udf依赖
wget https://archive.apache.org/dist/doris/1.2/1.2.4.1-rc01/apache-doris-dependencies-1.2.4.1-bin-x86_64.tar.xz
# 解压
tar -xvf apache-doris-dependencies-1.2.4.1-bin-x86_64.tar.xz
# 替换文件
mv apache-doris-dependencies-1.2.4.1-bin-x86_64/* ./启动be
# 进入be容器
docker exec -it doris-fe /bin/bash
# 进入lib目录 --daemon 后台运行，如果不放心先去掉，前台运行看日志
/opt/apache-doris/be/bin/start_be.sh --daemon注册be
# 进入一台有mysql的服务器，端口9093，账号root，默认密码是空
mysql -h ${fe的ip} -P9030 -uroot
# 注册be
ALTER SYSTEM ADD BACKEND "${be的ip}:9050";
# 查看be Alive属性如果是true就ok了
show PROC '/backends';MySQL [(none)]> show PROC '/backends';
+-----------+-----------------+---------------+----------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| BackendId | Cluster | IP | HostName | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole |
+-----------+-----------------+---------------+----------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| 10003 | default_cluster | 172.16.10.142 | hdp04 | 9050 | 9060 | 8041 | 8060 | 2023-06-15 08:42:32 | 2023-06-15 08:42:45 | true | false | false | 0 | 0.000 | 713.640 GB | 899.561 GB | 20.67 % | 20.67 % | 0.000 | {"location" : "default"} | | doris-1.2.4-1-Unknown | {"lastSuccessReportTabletsTime":"2023-06-15 08:42:39","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
+-----------+-----------------+---------------+----------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+doris连接测试
账号root，默认密码为空
本文链接：https://my.lmcjl.com/post/13537.html
展开阅读全文
4 评论