1. 编程号首页

Apache Ranger 编译安装

编程小号 • • 未分类

Apache Ranger 编译安装开源开发者笔记：DevOps，微服务，分布式，大数据，高可用，区块链，白皮书，算法，黑客，设计模式，面试题。求 star⭐️ Apache Ranger是大数据领域的一个集中式安全管理框架，目的是通过制定策略（policies）实现对Hadoop组件的集中式安全管理。用户可以通…

开源开发者笔记：DevOps，微服务，分布式，大数据，高可用，区块链，白皮书，算法，黑客，设计模式，面试题。求 star⭐️

Apache Ranger是大数据领域的一个集中式安全管理框架，目的是通过制定策略（policies）实现对Hadoop组件的集中式安全管理。用户可以通过Ranger实现对集群中数据的安全访问。

环境

框架 版本
Ubuntu 16.04
JAVA OpenJDK-8-jdk
Python 2.7
Maven 3.6.3
Ranger 2.0.0
Hadoop 3.1.3
Solr 8.5.2

源码编译

环境准备

  • 下载 Ranger 源码
wget https://downloads.apache.org/ranger/2.0.0/apache-ranger-2.0.0.tar.gz
或使用镜像
wget https://mirrors.tuna.tsinghua.edu.cn/apache/ranger/2.0.0/apache-ranger-2.0.0.tar.gz
  • 下载 Maven
wget https://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
  • 安装 java、python及c++编译环境
apt install openjdk-8-jdk
apt install python2.7
apt install gcc g++
ln -s /usr/bin/python2.7 /usr/bin/python
  • 配置环境变量
export MAVEN_HOME=/opt/app/apache-maven-3.6.3
export PATH=$PATH:$MAVEN_HOME/bin
export MAVEN_OPTS=-Xmx2048m
  • 使用 ali maven 源

/opt/app/apache-maven-3.6.2/conf/settings.xml

<mirrors>
   <mirror>
     <id>nexus-aliyun</id>
     <mirrorOf>central</mirrorOf>
     <name>Nexus aliyun</name>
     <url>http://maven.aliyun.com/nexus/content/groups/public</url>
   </mirror>

   <mirror>
     <id>CN</id>
     <name>OSChina Central</name>
     <url>http://maven.oschina.net/content/groups/public/</url>
     <mirrorOf>central</mirrorOf>
   </mirror>

   <mirror>
     <id>alimaven</id>
     <mirrorOf>central</mirrorOf>
     <name>aliyun maven</name>
     <url>https://maven.aliyun.com/nexus/content/repositories/central/</url>
   </mirror>

   <mirror>
     <id>jboss-public-repository-group</id>
     <mirrorOf>central</mirrorOf>
     <name>JBoss Public Repository Group</name>
     <url>https://repository.jboss.org/nexus/content/groups/public</url>
   </mirror>
</mirrors>

编译

tar xf apache-ranger-2.0.0.tar.gz
cd apache-ranger-2.0.0
mvn clean compile package assembly:assembly install -DskipTests -Drat.skip=true

编译成功打开 target 文件夹
apache-ranger-2.0.0/target# ll
total 1605220
drwxr-xr-x  5 root root      4096 Jun 29 11:04 ./
drwxr-xr-x 59 root root      4096 Jun 29 03:20 ../
drwxr-xr-x  2 root root      4096 Jun 29 03:20 antrun/
drwxr-xr-x  2 root root      4096 Jun 29 11:04 archive-tmp/
drwxr-xr-x  3 root root      4096 Jun 29 03:20 maven-shared-archive-resources/
-rw-r--r-- 1 root root 30 Jun 29 11:04 .plxarc
-rw-r--r-- 1 root root 248635508 Jun 29 11:01 ranger-2.0.0-admin.tar.gz
-rw-r--r-- 1 root root 249666712 Jun 29 11:02 ranger-2.0.0-admin.zip
-rw-r--r-- 1 root root 27787895 Jun 29 11:03 ranger-2.0.0-atlas-plugin.tar.gz
-rw-r--r-- 1 root root 27832012 Jun 29 11:03 ranger-2.0.0-atlas-plugin.zip
-rw-r--r-- 1 root root 31555304 Jun 29 11:04 ranger-2.0.0-elasticsearch-plugin.tar.gz
-rw-r--r-- 1 root root 31605841 Jun 29 11:04 ranger-2.0.0-elasticsearch-plugin.zip
-rw-r--r-- 1 root root 26638377 Jun 29 10:58 ranger-2.0.0-hbase-plugin.tar.gz
-rw-r--r-- 1 root root 26665982 Jun 29 10:58 ranger-2.0.0-hbase-plugin.zip
-rw-r--r-- 1 root root 23971392 Jun 29 10:58 ranger-2.0.0-hdfs-plugin.tar.gz
-rw-r--r-- 1 root root 23997623 Jun 29 10:58 ranger-2.0.0-hdfs-plugin.zip
-rw-r--r-- 1 root root 23825995 Jun 29 10:58 ranger-2.0.0-hive-plugin.tar.gz
-rw-r--r-- 1 root root 23854522 Jun 29 10:58 ranger-2.0.0-hive-plugin.zip
-rw-r--r-- 1 root root 39930681 Jun 29 10:59 ranger-2.0.0-kafka-plugin.tar.gz
-rw-r--r-- 1 root root 39983878 Jun 29 10:59 ranger-2.0.0-kafka-plugin.zip
-rw-r--r-- 1 root root 90982883 Jun 29 11:02 ranger-2.0.0-kms.tar.gz
-rw-r--r-- 1 root root 91106270 Jun 29 11:02 ranger-2.0.0-kms.zip
-rw-r--r-- 1 root root 28380704 Jun 29 10:58 ranger-2.0.0-knox-plugin.tar.gz
-rw-r--r-- 1 root root 28411022 Jun 29 10:58 ranger-2.0.0-knox-plugin.zip
-rw-r--r-- 1 root root 23940874 Jun 29 11:03 ranger-2.0.0-kylin-plugin.tar.gz
-rw-r--r-- 1 root root 23980053 Jun 29 11:03 ranger-2.0.0-kylin-plugin.zip
-rw-r--r-- 1 root root 34223 Jun 29 11:02 ranger-2.0.0-migration-util.tar.gz
-rw-r--r-- 1 root root 37740 Jun 29 11:02 ranger-2.0.0-migration-util.zip
-rw-r--r-- 1 root root 26388071 Jun 29 11:00 ranger-2.0.0-ozone-plugin.tar.gz
-rw-r--r-- 1 root root 26421136 Jun 29 11:00 ranger-2.0.0-ozone-plugin.zip
-rw-r--r-- 1 root root 40302042 Jun 29 11:04 ranger-2.0.0-presto-plugin.tar.gz
-rw-r--r-- 1 root root 40341626 Jun 29 11:04 ranger-2.0.0-presto-plugin.zip
-rw-r--r-- 1 root root 22232050 Jun 29 11:02 ranger-2.0.0-ranger-tools.tar.gz
-rw-r--r-- 1 root root 22248747 Jun 29 11:02 ranger-2.0.0-ranger-tools.zip
-rw-r--r-- 1 root root 42667 Jun 29 11:02 ranger-2.0.0-solr_audit_conf.tar.gz
-rw-r--r-- 1 root root 45636 Jun 29 11:02 ranger-2.0.0-solr_audit_conf.zip
-rw-r--r-- 1 root root 26964416 Jun 29 11:00 ranger-2.0.0-solr-plugin.tar.gz
-rw-r--r-- 1 root root 27010058 Jun 29 11:00 ranger-2.0.0-solr-plugin.zip
-rw-r--r-- 1 root root 23952732 Jun 29 11:03 ranger-2.0.0-sqoop-plugin.tar.gz
-rw-r--r-- 1 root root 23986050 Jun 29 11:03 ranger-2.0.0-sqoop-plugin.zip
-rw-r--r-- 1 root root 4081388 Jun 29 11:03 ranger-2.0.0-src.tar.gz
-rw-r--r-- 1 root root 6257752 Jun 29 11:03 ranger-2.0.0-src.zip
-rw-r--r-- 1 root root 37230628 Jun 29 10:59 ranger-2.0.0-storm-plugin.tar.gz
-rw-r--r-- 1 root root 37268719 Jun 29 10:59 ranger-2.0.0-storm-plugin.zip
-rw-r--r-- 1 root root 32772036 Jun 29 11:02 ranger-2.0.0-tagsync.tar.gz
-rw-r--r-- 1 root root 32782070 Jun 29 11:02 ranger-2.0.0-tagsync.zip
-rw-r--r-- 1 root root 16256778 Jun 29 11:02 ranger-2.0.0-usersync.tar.gz
-rw-r--r-- 1 root root 16280886 Jun 29 11:02 ranger-2.0.0-usersync.zip
-rw-r--r-- 1 root root 23953849 Jun 29 10:59 ranger-2.0.0-yarn-plugin.tar.gz
-rw-r--r-- 1 root root 23992362 Jun 29 10:59 ranger-2.0.0-yarn-plugin.zip
-rw-r--r-- 1 root root 5 Jun 29 11:04 version

编译时发生错误

  • 没安装c++编译环境
[INFO] Unix Native Authenticator .......................... FAILURE [ 1.387 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 08:51 min
[INFO] Finished at: 2020-06-030T12:00:43-02:00
[INFO] Final Memory: 96M/420M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:compile (default-compile) on project credValidator: Error executing command line. Exit code:127 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:compile (default-compile) on project credValidator: Error executing command line. Exit code:127
  • 没安装python
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (generate-version-annotation) on project ranger-util: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failonerror="true" executable="python">... @ 4:48 in /data/apache-ranger-2.0.0/ranger-util/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (generate-version-annotation) on project ranger-util: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec failonerror="true" executable="python">... @ 4:48 in /data/apache-ranger-2.0.0/ranger-util/target/antrun/build-main.xml

安装 ranger-admin

解压

tar zxvf ranger-2.0.0-admin.tar.gz -C /opt/ranger
cd /opt/ranger

修改 ranger-2.0.0-admin/install.properties

# 数据库连接
SQL_CONNECTOR_JAR=/opt/ranger/mysql-connector-java-8.0.18.jar
db_root_user=root
db_root_password=root
db_host=localhost

db_name=ranger
db_user=ranger
db_password=rangeradmin

# 日志审计
audit_store=solr
audit_solr_urls=http://localhost:6083/solr/ranger_audits
audit_solr_user=solr

执行 ranger-2.0.0-admin/setup.sh 进行安装

成功则输出。install.properties 配置中需要的 solr 、 mysql 、mysql驱动包如果没有可参考下面

2020-06-30 07:55:44,211  [I] Checking connection passed.
Installation of Ranger PolicyManager Web Application is completed.

执行 ranger-2.0.0-admin/set_globals.sh 配置软连接

执行 ranger-admin start 或者 ranger-2.0.0-admin/ews/ranger-admin-services.sh start 启动服务

浏览器访问 http://ip:6080/ admin/admin

Apache Ranger 编译安装

安装 solr

修改 ranger-2.0.0-admin/contrib/solr_for_audit_setup/install.properties

SOLR_INSTALL=true

JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
SOLR_DOWNLOAD_URL=https://mirrors.tuna.tsinghua.edu.cn/apache/lucene/solr/8.5.2/solr-8.5.2.tgz

SOLR_INSTALL_FOLDER=/data/solr   #安装目录，这个随意
SOLR_RANGER_HOME=/data/solr/ranger_audit_server
SOLR_RANGER_PORT=6083   #默认端口
SOLR_DEPLOYMENT=standalone  #部署模式(单节点部署)，还有solrcloud(集群)模式
SOLR_RANGER_DATA_FOLDER=/data/solr/ranger_audit_server/data   #数据存放目录

执行 ranger-2.0.0-admin/contrib/solr_for_audit_setup/setup.sh

Tue Jun 30 06:58:31 UTC 2020|INFO|Installed Solr in /opt/solr
Tue Jun 30 06:58:31 UTC 2020|INFO|Configuring standalone instance
Tue Jun 30 06:58:31 UTC 2020|INFO|Copying Ranger Audit Server configuration to /opt/solr/ranger_audit_server
Tue Jun 30 06:58:31 UTC 2020|INFO|Creating group solr
Tue Jun 30 06:58:31 UTC 2020|INFO|Creating user solr
Tue Jun 30 06:58:31 UTC 2020|INFO|Done configuring Solr for Apache Ranger Audit
Tue Jun 30 06:58:31 UTC 2020|INFO|Solr HOME for Ranger Audit is /opt/solr/ranger_audit_server
Tue Jun 30 06:58:31 UTC 2020|INFO|Data folder for Audit logs is /opt/solr/ranger_audit_server/data
Tue Jun 30 06:58:31 UTC 2020|INFO|To start Solr run /opt/solr/ranger_audit_server/scripts/start_solr.sh
Tue Jun 30 06:58:31 UTC 2020|INFO|To stop Solr run /opt/solr/ranger_audit_server/scripts/stop_solr.sh
Tue Jun 30 06:58:31 UTC 2020|INFO|After starting Solr for RangerAudit, it will listen at 6083. E.g http://weihai-2:6083
Tue Jun 30 06:58:31 UTC 2020|INFO|Configure Ranger to use the following URL http://weihai-2:6083/solr/ranger_audits
Tue Jun 30 06:58:31 UTC 2020|INFO| ** NOTE: If Solr is Secured then solrclient JAAS configuration has to be added to Ranger Admin and Ranger Plugin properties
Tue Jun 30 06:58:31 UTC 2020|INFO| ** Refer documentation on how to configure Ranger for audit to Secure Solr
########## Done ###################
Created file /opt/solr/ranger_audit_server/install_notes.txt with instructions to start and stop
###################################

启动/停止 solr

/opt/solr/ranger_audit_server/scripts/start_solr.sh
/opt/solr/ranger_audit_server/scripts/stop_solr.sh

安装 MySQL

  • 下载 mysql 驱动包 https://downloads.mysql.com/archives/c-j/

  • 下载 mysql-server

    • 官网下载
    https://dev.mysql.com/downloads/mysql/

    或

    • 更新APT源
    wget https://dev.mysql.com/get/mysql-apt-config_0.8.15-1_all.de
pkg -i mysql-apt-config_0.8.15-1_all.deb

apt search mysql-server
Sorting... Done
Full Text Search... Done
auth2db/xenial 0.2.5-2+dfsg-5ubuntu1 all
  Powerful and eye-candy IDS logger, log viewer and alert generator

mysql-server/unknown,now 8.0.20-1ubuntu16.04 amd64 [installed]
  MySQL Server meta package depending on latest version
  
  
apt install mysql-server

  • 配置 mysql

create user 'ranger'@'%' identified by 'rangeradmin';
create database ranger;
grant all privileges on ranger.* to 'ranger'@'%';
set global log_bin_trust_function_creators=TRUE;
SET GLOBAL innodb_lock_wait_timeout=500;

今天的文章Apache Ranger 编译安装分享到此就结束了，感谢您的阅读。

版权声明：本文内容由互联网用户自发贡献，该文观点仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 举报，一经查实，本站将立刻删除。
如需转载请保留出处：https://bianchenghao.cn/23053.html

赞 (0)
编程小号编程小号
0

相关推荐

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注