Skip to content

Commit 17a7099

Browse files
alldatafounderalldatafounder
committed
feature(alldatadc): modify structure
1 parent 4bae91d commit 17a7099

19 files changed

+35
-447
lines changed

‎README.md

+12-223
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,16 @@
11
# AllData 一站式大数据平台
22

33
<br/>
4-
<p>
5-
<a href="https://github.com/alldatacenter/alldata/commits/master" target="_blank">
6-
<img width=200 height="40" src="https://img.shields.io/github/last-commit/authorwlh/alldata.svg">
7-
</a>
8-
<a href="https://github.com/alldatacenter/alldata/issues?q=is%3Aissue+is%3Aopen+label%3Abug" target="_blank">
9-
<img width=200 height="40" src="https://img.shields.io/github/issues/authorwlh/alldata/bug.svg?label=bugs&color=red">
10-
</a>
11-
<a href="https://github.com/alldatacenter/alldata/issues?q=is%3Aissue+is%3Aopen+label%3Aenhancement" target="_blank">
12-
<img width=200 height="40" src="https://img.shields.io/github/issues/authorwlh/alldata/enhancement.svg?label=enhancements&color=cyan">
13-
</a>
14-
<a href="https://github.com/alldatacenter/alldata/blob/master/LICENSE" target="_blank">
15-
<img width=200 height="40" src="https://img.shields.io/github/license/authorwlh/alldata.svg">
16-
</a>
17-
</p>
4+
5+
<a href="https://github.com/alldatadc/github-readme-stats">
6+
<img width="1215" align="center" src="https://github-readme-stats.vercel.app/api?username=alldatadc&hide=stars&show_icons=true" />
7+
</a>
8+
<br/>
9+
<br/>
10+
<a href="https://github.com/alldatacenter/github-readme-stats">
11+
<img width="1215" align="center" src="https://github-readme-stats.vercel.app/api/pin/?username=alldatacenter&repo=alldata" />
12+
</a>
13+
1814

1915
## [中文](https://github.com/alldatacenter/alldata/blob/master/README_ZH.md) | English
2016
## [原型体验预览版](https://orgnext.modao.cc/app/HhitGZQTr954c7Ug8XBvAY) | [Documentation](https://github.com/alldatacenter/alldata/blob/master/README.md) | [Installation](#installation) | [Architecture](#architecture) | [Integration](#integration) | [Community](#community)
@@ -553,31 +549,10 @@
553549
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/171133364-8e4a8e84-c9f9-456c-9f33-c90b90cf54e4.png">
554550
<br/>
555551

556-
### Flink1.16 OGG-JSON解析
557-
558-
<br/>
559-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/174279501-3774b028-8252-466d-9ba5-9a02ba677c63.png">
560-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/174279566-d528dde0-e9af-4f4d-9864-318f41f82ead.png">
561-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/174279777-24966e66-5b93-4e2b-bf4b-2e8ead49380c.png">
562-
S3 Hudi成功写入
563-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/174279203-b5a39a9f-05e1-466b-a12f-3f6cf9ed3033.png">
564-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/174279028-b53d45f2-255d-48bd-b3aa-0df8f0b9698c.png">
565-
566-
567-
568-
## 1、数据平台
569-
570-
> AllData is one of the few open source big data platform projects on Github.
571-
> It will develop into a successful solution to solve a series of problems in big data e-commerce scenarios.
572-
> It will also become a general big data base for other developers to use and Contribution, my original intention is to create a product that is useful to society.
573-
574-
575-
## 2、商城展示
552+
## 商城展示
576553
<br>
577554
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160219586-e2e190fa-21f6-4f87-bbbc-7cdd6ecc625a.png">
578555
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160219649-d6a0b18c-7e54-4f8d-8d95-17777cdbe9d9.png">
579-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160219678-10998762-6266-42b7-8fa4-0a725f215881.png">
580-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160219693-84c4a0de-d4ab-4ad9-bc55-202d0ec0afb0.png">
581556
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160221446-24d9438d-703c-4d17-880e-5d34d0f8d229.png">
582557
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160221463-772477c8-f996-45df-ab74-9e7a179adc81.png">
583558

@@ -598,197 +573,11 @@ S3 Hudi成功写入
598573
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220106-0341a2f4-b4df-4d2b-9ec1-b0f10affd22d.png">
599574
<br/>
600575

601-
### 3、数据来源
602-
~~~markdown
603-
商城前台：
604-
mall-shopping-app: 商城App
605-
mall-shopping-app-service: 商城App服务
606-
mall-shopping-wc: 商城小程序
607-
mall-shopping-mobile: 商城前台
608-
mall-shopping-pc: 商城pc端
609-
pcAdminService: 商城pc端服务
610-
mobileService: 商城前台服务（小程序和前台接入此接口）
611-
商城后台：
612-
mall-admin-web: 商城后台
613-
pcAdminService: 商城后台服务
614-
~~~
615-
616-
### 4、数据收集
617-
~~~markdown
618-
log-collect-server:
619-
服务端日志收集系统
620-
log-collect-client:
621-
支持各app集成的客户端SDK，负责收集app客户端数据；
622-
data-import-export:
623-
基于DataX实现数据集成(导入导出)
624-
data-spider:
625-
爬虫平台支持可配置的爬取公网数据的任务开发；
626-
~~~
627-
<br/>
628-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220310-25525dad-03ea-42ed-acd1-c5625a7c809f.png">
629-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220316-d59c47f1-ed0a-44fe-9ca8-ae0fe85c904f.png">
630-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220323-db53951d-1956-4c83-8642-b086b8f4d90c.png">
631-
<br/>
632-
633-
### 5、数据存储
634-
~~~markdown
635-
分布式文件系统：hdfs
636-
分布式数据库：hbase、mongodb、elasticsearch
637-
分布式内存存储系统：redis
638-
~~~
639-
640-
### 6、数据计算
641-
~~~markdown
642-
compute-mr（离线计算）: Hive、MR
643-
compute-realtime（流计算）: storm、flink
644-
multi-dimension-analysis（多维度分析）: kylin, spark
645-
~~~
646-
647-
### 7、数据开发
648-
~~~markdown
649-
task-schedular: 任务调度
650-
task-ops: 任务运维
651-
~~~
652-
<br/>
653-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220366-7c913374-c5fe-4488-be82-1339c1441227.png">
654-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220373-c775d811-3162-4719-86b1-7dfe53ac4326.png">
655-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220381-c1cc8344-e16c-4d03-8434-b9d1ef43470c.png">
656-
<br/>
657-
658-
### 8、数据产品
659-
~~~markdown
660-
data-face: 数据可视化
661-
data-insight: 用户画像分析
662-
~~~
663-
<br/>
664-
665-
### 9、数据应用
666-
~~~markdown
667-
system-recommender: 推荐
668-
system-ad: 广告
669-
system-search: 搜索
670-
system-anti-cheating: 反作弊
671-
system-report-analysis: 报表分析
672-
system-elk: ELK日志系统，实现日志搜索平台
673-
system-apm: skywalking监控平台
674-
system-deploy: k8s，scala，playframework，docker打包平台。
675-
job-schedule: 任务提交平台
676-
~~~
677-
678-
<br/>
679-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220425-afc819af-e8f9-4fbb-811d-c0b67cccf7d0.png">
680-
<br/>
681576

682577
## Installation
683578

684-
### 10、启动配置教程
685-
686-
10.1 启动前，打包dubbo-servie项目，进入dubbo目录，
687-
688-
执行mvn clean package -DskipTests=TRUE打包，然后执行mvn install.
689-
690-
10.2 启动dubbo项目，配置tomcat端口为8091
691-
692-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220455-45898c53-0de6-4a06-80b0-ae7e758b9457.png">
693-
694-
695-
10.3 启动商城项目的多个子系统
696-
697-
后台：访问http://localhost:8090
698-
699-
10.3.1、前端：启动mall-admin-web项目，进入项目目录，执行npm install，然后执行npm run dev；
700-
701-
10.3.2、后端：启动pcAdminService/mall-admin-search项目，
702-
703-
配置tomcat端口为8092，接着启动pcManage项目，tomcat端口配置为8093；
704-
705-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220467-283a7964-27c1-4184-9ece-778e87fc38f7.png">
706-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220472-68a6d9a4-e295-4b86-a9e6-75b53f821d52.png">
707-
708-
709-
前台：小程序手机预览，移动端访问：http://localhost:6255
710-
711-
10.3.3、小程序和移动端
712-
713-
10.3.3.1、前端：商城小程序，启动mall-shopping-wc项目，
714-
715-
安装微信开发者工具，配置开发者key和secret，
716-
717-
使用微信开发者工具导入即可，然后点击编译，可以手机预览使用。
718-
719-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220487-dace0ed7-c4e5-4a17-88d9-983e44c89ce5.png">
720-
721-
722-
10.3.3.2、前端：商城移动端，启动mall-shopping-mobile，
723-
724-
进入项目目录，执行npm install和npm run dev；
725-
726-
10.3.3.3、后端：小程序和移动端用的是同一个后台服务，
727-
728-
启动mobileService项目，进入项目目录，配置tomcat端口8094
729-
730-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220500-7c6b9097-7a82-4f23-95be-eda9c8f9eee5.png">
731-
732-
733-
10.3.4、商城PC端 访问http://localhost:8099
734-
735-
10.3.4.1、前端：启动mall-shopping-pc项目，
736-
737-
进入项目目录，执行npm install和npm run dev；
738-
739-
10.3.4.2、后端：启动pcAdminService项目，配置tomcat端口为8095；
740-
741-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220506-688f51cc-1b3d-46a9-ad3e-ec033ee69562.png">
742-
743-
744-
745-
### 11、DevOPS
746-
747-
11.1 容器化部署system-deploy
748-
749-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220577-3409bf17-d828-4905-b10e-7e9ba20e544e.png">
750-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220585-e245f8bc-0821-47ab-b748-808c884cd4de.png">
751-
752-
11.2、自动化运维平台system-devops
753-
754-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220594-40e4c4ba-123b-4ecc-98f3-345e0cd0843b.png">
755-
756-
11.3、使用Kong作为调用中心网关入口system-api-gateway
757-
758-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220600-6b3443ef-cf30-433a-bf28-a03147c0b069.png">
759-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220611-5f8327b4-0ca9-457c-b623-8f2026e81f59.png">
760-
761-
762-
11.4、日志中心system-elk
763-
764-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220621-bd22df74-e11c-4a34-86f1-35fd17b881c6.png">
765-
766-
11.5、告警平台system-alarm-platform
767-
768-
11.6 监控系统
769-
770-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220634-1e1bdf37-26e5-4c35-b56f-ee55af4dcd45.png">
771-
772-
11.7 数据采集
773-
774-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220653-6264e6a7-09a2-4811-a99f-99b118433208.png">
775-
776-
11.8 数据展示
777-
778-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220670-1b30e201-28b0-4fcc-8f0e-9aea8154f43a.png">
779-
780-
781-
11.9 监控中心system-apm
782-
783-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220682-24063605-59b8-43ad-928b-d1ce8be603b2.png">
784-
785-
786-
11.10 使用Apollo作为配置中心system-config
787-
788-
<img width="1215" alt="image" src="https://user-images.githubusercontent.com/20246692/160220690-7fc23e65-ad53-4f5f-a0f2-aa55e649f500.png">
789-
790579
## Community
791580

792-
## 12、近期进行社区修整，闭关期间，仅保留微信群
581+
## 近期进行社区修整，闭关期间，仅保留微信群
793582

794583
> 联系作者: https://docs.qq.com/doc/DVFVMYUp6cFhSRVJs

0 commit comments

Comments
 (0)