Initial commit: Gloria project with Confluence API integration

This commit is contained in:
qichi.liang
2026-02-04 13:48:44 +08:00
commit a1492e968a
25 changed files with 4013 additions and 0 deletions

12
.env.example Normal file
View File

@@ -0,0 +1,12 @@
# Confluence 配置
CONFLUENCE_BASE_URL=https://confluence.westwell-lab.com
CONFLUENCE_TOKEN=your_token_here
ROOT_PAGE_ID=137446574
# 缓存配置
CACHE_DIR=./cache
CACHE_TTL=3600
# 应用配置
APP_HOST=0.0.0.0
APP_PORT=8000

65
.gitignore vendored Normal file
View File

@@ -0,0 +1,65 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# Virtual environments
venv/
env/
ENV/
env.bak/
venv.bak/
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# Environment variables
.env
.env.local
.env.*.local
# Cache
cache/
*.cache
.pytest_cache/
# Logs
*.log
logs/
# Database
*.db
*.sqlite3
# OS
.DS_Store
Thumbs.db
# Test files
test_*.py
*_test.py
tests/

314
README.md Normal file
View File

@@ -0,0 +1,314 @@
# 福州江阴实船作业统计可视化系统
基于 FastAPI + ECharts 的 Confluence 数据采集与可视化报表系统。
## 🎯 项目目标
自动采集 Confluence "福州江阴实船作业统计" 页面及其子页面的表格数据,
生成面向团队的 Web 可视化报表。
## 🏗️ 系统架构
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Confluence │────▶│ Data Fetcher │────▶│ Data Parser │
│ API │ │ (Confluence │ │ (Table Extract)│
│ │ │ Client) │ │ │
└─────────────────┘ └──────────────────┘ └─────────────────┘
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Web Frontend │◄────│ FastAPI │◄────│ Data Store │
│ (ECharts Viz) │ │ REST API │ │ (JSON/Cache) │
└─────────────────┘ └──────────────────┘ └─────────────────┘
```
## 🛠️ 技术栈
- **后端**: Python 3.10+, FastAPI, Uvicorn
- **前端**: HTML5, JavaScript, ECharts 5.x
- **数据采集**: requests, BeautifulSoup4, pandas
- **缓存**: 文件系统缓存JSON
## 📡 Confluence API 集成
### API 端点
| 端点 | 描述 |
|------|------|
| `GET /rest/api/content/{id}` | 获取页面内容 |
| `GET /rest/api/content/{id}/child/page` | 获取子页面列表 |
### 认证方式
使用 Bearer Token 认证:
```python
headers = {
"Authorization": "Bearer {token}",
"Content-Type": "application/json"
}
```
### 页面树遍历
```python
async def get_page_tree(page_id: str) -> List[PageNode]:
"""
递归获取页面及其所有子页面
"""
children = await fetch_children(page_id)
tree = []
for child in children:
node = PageNode(
id=child['id'],
title=child['title'],
url=child['_links']['webui'],
children=await get_page_tree(child['id'])
)
tree.append(node)
return tree
```
### 表格数据提取
```python
def extract_table_data(html_content: str) -> pd.DataFrame:
"""
从 Confluence 页面 HTML 中提取表格数据
"""
soup = BeautifulSoup(html_content, 'html.parser')
tables = soup.find_all('table', {'class': 'confluenceTable'})
if not tables:
return pd.DataFrame()
# 使用 pandas 解析第一个表格
df = pd.read_html(str(tables[0]))[0]
return df
```
## 📊 数据模型
### 页面元数据
```python
class PageNode(BaseModel):
id: str
title: str
url: str
children: List['PageNode'] = []
data: Optional[ShipData] = None
```
### 船舶报告数据
```python
class ShipData(BaseModel):
ship_code: str # 如 "FZ 361#"
location: str # "FZ" = 福州
ship_number: str # "361#"
report_date: date # 报告日期
raw_data: Dict # 原始表格数据
```
### 月度统计
```python
class MonthlyStats(BaseModel):
month: str # "2026.02"
page_id: str
total_ships: int
ships: List[ShipData]
summary: Dict # 汇总统计
```
## 🔌 FastAPI API 规范
### 端点列表
| 方法 | 端点 | 描述 |
|------|------|------|
| GET | `/api/health` | 健康检查 |
| GET | `/api/pages/tree` | 获取完整页面树 |
| GET | `/api/pages/{page_id}` | 获取指定页面数据 |
| GET | `/api/monthly/{month}` | 获取月度统计 |
| GET | `/api/statistics/summary` | 获取整体汇总 |
| POST | `/api/refresh` | 强制刷新缓存 |
### 响应示例
**获取页面树**
```json
{
"root_page": {
"id": "137446574",
"title": "福州江阴实船作业统计",
"children": [
{
"id": "137446576",
"title": "2025.06 实船作业统计",
"children": []
},
{
"id": "161630736",
"title": "2026.02 实船作业统计",
"children": [
{
"id": "161630738",
"title": "FZ 361#实船报告2026.02.01",
"children": []
}
]
}
]
}
}
```
**获取统计数据**
```json
{
"month": "2026.02",
"total_ships": 15,
"ships": [
{
"ship_code": "FZ 361#",
"report_date": "2026-02-01",
"data": {
"作业量": "1000吨",
"作业时长": "8小时"
}
}
]
}
```
## 📈 可视化方案
### 推荐图表类型
1. **月度趋势图**(折线图)
- X轴月份2025.06 - 2026.02
- Y轴作业次数/作业量
2. **船次分布图**(饼图/环形图)
- 各船次出现频次分布
3. **日历热力图**
- 展示作业日期分布密度
4. **船次效率对比**(横向柱状图)
- 比较各船次的平均作业量
5. **汇总仪表板**
- KPI 卡片:总船次、月均船次、活跃船次数
### 页面布局
```
┌─────────────────────────────────────┐
│ 关键指标卡片区域 │
├─────────────────┬─────────────────┤
│ 月度趋势图 │ 船次分布图 │
├─────────────────┴─────────────────┤
│ 日历热力图 │
├─────────────────┬─────────────────┤
│ 船次对比图 │ 数据明细表 │
└─────────────────┴─────────────────┘
```
### ECharts 示例
```javascript
// 月度趋势图
const trendChart = echarts.init(document.getElementById('trend'));
const option = {
title: { text: '月度实船作业趋势' },
xAxis: {
type: 'category',
data: ['2025.06', '2025.07', '2025.08', '2025.09',
'2025.10', '2025.11', '2025.12', '2026.01', '2026.02']
},
yAxis: { type: 'value', name: '作业次数' },
series: [{
data: [12, 15, 18, 20, 22, 19, 25, 21, 15],
type: 'line',
smooth: true,
areaStyle: {}
}]
};
trendChart.setOption(option);
```
## 🚀 部署指南
### 环境配置
创建 `.env` 文件:
```bash
CONFLUENCE_BASE_URL=https://confluence.westwell-lab.com
CONFLUENCE_TOKEN=your_token_here
ROOT_PAGE_ID=137446574
CACHE_DIR=./cache
```
### 安装依赖
```bash
pip install fastapi uvicorn requests beautifulsoup4 pandas python-dotenv
```
### 运行服务
```bash
# 开发模式
uvicorn main:app --reload
# 生产模式
uvicorn main:app --host 0.0.0.0 --port 8000
```
### 访问应用
- API 文档: `http://localhost:8000/docs`
- 可视化页面: `http://localhost:8000/static/index.html`
## 📁 项目结构
```
project/
├── main.py # FastAPI 主应用
├── config.py # 配置管理
├── confluence/
│ ├── __init__.py
│ ├── client.py # Confluence API 客户端
│ └── parser.py # 数据解析器
├── models/
│ ├── __init__.py
│ └── schemas.py # Pydantic 数据模型
├── services/
│ ├── __init__.py
│ ├── cache.py # 缓存管理
│ └── data_service.py # 数据服务
├── static/
│ ├── index.html # 可视化页面
│ ├── css/
│ └── js/
│ └── charts.js # ECharts 配置
├── cache/ # 缓存文件目录
├── .env # 环境变量
└── README.md
```
## 🔧 后续扩展
- [ ] 数据导出功能Excel/PDF
- [ ] 定时自动同步
- [ ] 邮件通知
- [ ] 历史数据对比分析
- [ ] 移动端适配
## 📝 待确认事项
1. **表格字段**:抓取示例页面后确定需要提取的具体字段
2. **更新频率**:是否需要定时同步或仅手动刷新
3. **权限控制**:是否需要用户登录功能

95
check_2026_01.py Normal file
View File

@@ -0,0 +1,95 @@
"""直接从Confluence API获取2026.01页面的子页面"""
import asyncio
import sys
sys.path.insert(0, '.')
from confluence.client import ConfluenceClient
async def check_2026_01_children():
"""检查2026.01页面的所有子页面"""
print("=" * 60)
print("🔍 重新获取2026.01页面的子页面")
print("=" * 60)
client = ConfluenceClient()
# 2026.01 页面ID: 159049201
page_id = "159049201"
try:
children = await client.get_children(page_id)
print(f"\n📄 找到 {len(children)} 个子页面\n")
# 按标题排序
sorted_children = sorted(children, key=lambda x: x.get('title', ''))
print("所有子页面标题:")
print("-" * 60)
for i, child in enumerate(sorted_children, 1):
title = child.get('title', 'N/A')
child_id = child.get('id', 'N/A')
print(f"{i:3d}. {title} (ID: {child_id})")
# 检查是否包含353-360
if any(f'FZ {n}#' in title for n in range(353, 361)):
print(f" ⭐ 找到353-360范围内的船次!")
# 特别查找353-360
print("\n" + "=" * 60)
print("🔎 特别查找 FZ 353# 到 FZ 360#:")
print("=" * 60)
found_ships = []
for child in children:
title = child.get('title', '')
if 'FZ ' in title:
try:
# 提取船次数字
import re
match = re.search(r'FZ\s+(\d+)#', title)
if match:
num = int(match.group(1))
if 353 <= num <= 360:
found_ships.append((num, title, child.get('id')))
except:
pass
if found_ships:
print(f"\n✅ 找到 {len(found_ships)} 艘船 (353-360#):")
for num, title, child_id in sorted(found_ships):
print(f" {title} (ID: {child_id})")
else:
print("\n❌ 未找到353-360#范围内的船次")
# 显示350#之后的船次
print("\n" + "=" * 60)
print("📊 350#之后的所有船次:")
print("=" * 60)
ships_after_350 = []
for child in children:
title = child.get('title', '')
if 'FZ ' in title:
try:
import re
match = re.search(r'FZ\s+(\d+)#', title)
if match:
num = int(match.group(1))
if num >= 350:
ships_after_350.append((num, title, child.get('id')))
except:
pass
ships_after_350.sort()
for num, title, child_id in ships_after_350:
print(f" {title}")
except Exception as e:
print(f"\n❌ 错误: {e}")
import traceback
traceback.print_exc()
finally:
await client.close()
if __name__ == "__main__":
asyncio.run(check_2026_01_children())

63
check_all_children.py Normal file
View File

@@ -0,0 +1,63 @@
"""获取2026.01页面的所有子页面(包括分页)"""
import asyncio
import sys
sys.path.insert(0, '.')
from confluence.client import ConfluenceClient
async def get_all_children():
"""获取所有子页面,处理分页"""
print("=" * 60)
print("🔍 获取2026.01所有子页面(处理分页)")
print("=" * 60)
client = ConfluenceClient()
page_id = "159049201"
try:
# 获取子页面,增加限制
children = await client.get_children(page_id, limit=200)
print(f"\n📄 找到 {len(children)} 个子页面\n")
# 按标题排序
sorted_children = sorted(children, key=lambda x: x.get('title', ''))
# 查找353-360
found_ships = []
for child in sorted_children:
title = child.get('title', '')
child_id = child.get('id', '')
if 'FZ ' in title:
try:
import re
match = re.search(r'FZ\s+(\d+)#', title)
if match:
num = int(match.group(1))
if 350 <= num <= 365:
found_ships.append((num, title, child_id))
except:
pass
if found_ships:
print(f"350-365# 船次列表:\n")
for num, title, child_id in sorted(found_ships):
print(f" {title} (ID: {child_id})")
else:
print("❌ 未找到350-365#范围内的船次")
# 显示最后10条
print(f"\n最后10个子页面:")
print("-" * 60)
for child in sorted_children[-10:]:
print(f" {child.get('title')} (ID: {child.get('id')})")
except Exception as e:
print(f"\n❌ 错误: {e}")
import traceback
traceback.print_exc()
finally:
await client.close()
if __name__ == "__main__":
asyncio.run(get_all_children())

69
check_nov_11_18.py Normal file
View File

@@ -0,0 +1,69 @@
"""查找2025年11月11日和18日的船次"""
import asyncio
import sys
sys.path.insert(0, '.')
from confluence.client import ConfluenceClient
async def check_nov_dates():
"""检查11月11日和18日的船次"""
print("=" * 60)
print("🔍 查找2025年11月11日和18日的船次")
print("=" * 60)
client = ConfluenceClient()
# 2025.11月度统计页面ID: 151907843
page_id = "151907843"
try:
children = await client.get_children(page_id, limit=200)
print(f"\n📄 找到 {len(children)} 个子页面\n")
# 查找包含11月11日或18日的页面
target_dates = ['2025.11.11', '2025.11.18']
print(f"查找日期: {target_dates}")
print("-" * 60)
found_ships = []
for child in children:
title = child.get('title', '')
child_id = child.get('id', '')
# 检查是否包含目标日期
for date_str in target_dates:
if date_str in title:
found_ships.append((title, child_id))
print(f"✅ 找到: {title} (ID: {child_id})")
break
if not found_ships:
print("❌ 未找到11月11日或18日的船次")
# 列出所有11月的船次
print(f"\n📅 2025年11月所有船次部分列表:")
print("-" * 60)
nov_ships = []
for child in children:
title = child.get('title', '')
if '2025.11' in title or '2025.11' in title:
nov_ships.append(title)
nov_ships.sort()
for i, title in enumerate(nov_ships[:15], 1):
print(f" {i}. {title}")
if len(nov_ships) > 15:
print(f" ... 还有 {len(nov_ships) - 15}")
except Exception as e:
print(f"\n❌ 错误: {e}")
import traceback
traceback.print_exc()
finally:
await client.close()
if __name__ == "__main__":
asyncio.run(check_nov_dates())

58
check_page_161628587.py Normal file
View File

@@ -0,0 +1,58 @@
"""获取指定Confluence页面的详细内容"""
import asyncio
import sys
sys.path.insert(0, '.')
from confluence.client import ConfluenceClient
from confluence.parser import DataParser
async def check_specific_page():
"""检查用户提供的页面"""
print("=" * 60)
print("🔍 检查页面: 161628587")
print("=" * 60)
client = ConfluenceClient()
page_id = "161628587"
try:
# 获取页面基本信息
page_info = await client.get_page(page_id)
if page_info:
print(f"\n📄 页面标题: {page_info.get('title', 'N/A')}")
print(f"🆔 页面ID: {page_info.get('id', 'N/A')}")
print(f"📅 版本: {page_info.get('version', {}).get('number', 'N/A')}")
# 获取页面内容
html_content = await client.get_page_content(page_id)
if html_content:
print(f"\n📃 HTML内容长度: {len(html_content)} 字符")
# 解析表格
df = DataParser.extract_table_data(html_content)
if df is not None:
print(f"\n📊 提取的表格: {len(df)} 行 x {len(df.columns)}")
print(f"\n表格内容:")
print(df.to_string())
# 转换为字典
table_dict = DataParser.dataframe_to_dict(df)
print(f"\n📋 转换后的字典:")
for key, value in table_dict.items():
print(f" {key}: {value}")
else:
print("\n⚠️ 未找到表格")
print(f"\nHTML片段 (前500字符):")
print(html_content[:500])
else:
print("\n❌ 无法获取页面内容")
except Exception as e:
print(f"\n❌ 错误: {e}")
import traceback
traceback.print_exc()
finally:
await client.close()
if __name__ == "__main__":
asyncio.run(check_specific_page())

56
check_user_pages.py Normal file
View File

@@ -0,0 +1,56 @@
"""检查用户提供的页面"""
import asyncio
import sys
sys.path.insert(0, '.')
from confluence.client import ConfluenceClient
from confluence.parser import DataParser
async def check_pages():
"""检查用户提供的两个页面"""
print("=" * 60)
print("🔍 检查用户提供的页面")
print("=" * 60)
client = ConfluenceClient()
pages_to_check = [
("159032145", "Page 1"),
("159049306", "Page 2")
]
for page_id, label in pages_to_check:
print(f"\n📄 {label} (ID: {page_id})")
print("-" * 60)
try:
# 获取页面信息
page_info = await client.get_page(page_id)
if page_info:
print(f"标题: {page_info.get('title', 'N/A')}")
# 获取内容
html_content = await client.get_page_content(page_id)
if html_content:
df = DataParser.extract_table_data(html_content)
if df is not None:
table_dict = DataParser.dataframe_to_dict(df)
print(f"船次: {table_dict.get('船次', 'N/A')}")
print(f"船名: {table_dict.get('船名', 'N/A')}")
print(f"作业时间: {table_dict.get('作业时间', 'N/A')}")
print(f"TEU: {table_dict.get('作业箱量 (TEU)', 'N/A')}")
print(f"Moves: {table_dict.get('作业循环 (move)', 'N/A')}")
else:
print("未找到表格")
else:
print("无法获取内容")
else:
print("页面不存在")
except Exception as e:
print(f"错误: {e}")
await client.close()
if __name__ == "__main__":
asyncio.run(check_pages())

32
config.py Normal file
View File

@@ -0,0 +1,32 @@
from typing import Optional
from pydantic_settings import BaseSettings
from functools import lru_cache
class Settings(BaseSettings):
"""应用配置"""
# Confluence 配置
CONFLUENCE_BASE_URL: str = "https://confluence.westwell-lab.com"
CONFLUENCE_TOKEN: str = ""
ROOT_PAGE_ID: str = "137446574"
# 缓存配置
CACHE_DIR: str = "./cache"
CACHE_TTL: int = 3600 # 缓存有效期(秒)
# 应用配置
APP_HOST: str = "0.0.0.0"
APP_PORT: int = 8000
class Config:
env_file = ".env"
case_sensitive = True
@lru_cache()
def get_settings() -> Settings:
"""获取配置(单例)"""
return Settings()
settings = get_settings()

0
confluence/__init__.py Normal file
View File

194
confluence/client.py Normal file
View File

@@ -0,0 +1,194 @@
import httpx
from typing import List, Dict, Any, Optional
from config import settings
import asyncio
class ConfluenceClient:
"""Confluence API 客户端"""
def __init__(self):
self.base_url = settings.CONFLUENCE_BASE_URL.rstrip('/')
self.token = settings.CONFLUENCE_TOKEN
self.headers = {
"Authorization": f"Bearer {self.token}",
"Content-Type": "application/json",
"Accept": "application/json"
}
self.client = httpx.AsyncClient(
headers=self.headers,
timeout=30.0,
follow_redirects=True
)
async def close(self):
"""关闭 HTTP 客户端"""
await self.client.aclose()
async def get_page(self, page_id: str) -> Optional[Dict[str, Any]]:
"""
获取页面基本信息
Args:
page_id: 页面 ID
Returns:
页面信息字典
"""
url = f"{self.base_url}/rest/api/content/{page_id}"
try:
response = await self.client.get(url)
response.raise_for_status()
return response.json()
except httpx.HTTPStatusError as e:
print(f"获取页面失败: {e.response.status_code} - {e.response.text}")
return None
except Exception as e:
print(f"请求异常: {e}")
return None
async def get_page_content(self, page_id: str) -> Optional[str]:
"""
获取页面 HTML 内容
Args:
page_id: 页面 ID
Returns:
HTML 内容字符串
"""
url = f"{self.base_url}/rest/api/content/{page_id}"
params = {
"expand": "body.styled_view"
}
try:
response = await self.client.get(url, params=params)
response.raise_for_status()
data = response.json()
# 提取 HTML 内容(优先从 styled_view 获取)
body = data.get('body', {})
styled_view = body.get('styled_view', {})
content = styled_view.get('value', '')
# 如果 styled_view 为空,尝试备选方案
if not content:
print(f"页面 {page_id} 的 styled_view 内容为空,尝试获取 storage 格式...")
# 尝试获取 body.storage
storage = body.get('storage', {})
content = storage.get('value', '')
if not content:
# 尝试获取 view
view = body.get('view', {})
content = view.get('value', '')
if not content:
print(f"警告: 页面 {page_id} 的所有内容格式都为空")
else:
print(f"成功获取页面 {page_id} 内容,长度: {len(content)} 字符")
return content
except Exception as e:
print(f"获取页面内容失败: {e}")
return None
async def get_children(self, page_id: str, limit: int = 200) -> List[Dict[str, Any]]:
"""
获取子页面列表
Args:
page_id: 父页面 ID
limit: 返回数量限制
Returns:
子页面列表
"""
url = f"{self.base_url}/rest/api/content/{page_id}/child/page"
params = {
"limit": limit,
"expand": "_links"
}
children = []
start = 0
while True:
params['start'] = start
try:
response = await self.client.get(url, params=params)
response.raise_for_status()
data = response.json()
results = data.get('results', [])
if not results:
break
children.extend(results)
# 检查是否还有更多
size = data.get('size', 0)
if len(children) >= size:
break
start += limit
except Exception as e:
print(f"获取子页面失败: {e}")
break
return children
async def get_page_tree(self, page_id: str, max_depth: int = 5) -> Optional[Dict[str, Any]]:
"""
递归获取页面树
Args:
page_id: 起始页面 ID
max_depth: 最大递归深度
Returns:
页面树结构
"""
if max_depth <= 0:
return None
# 获取页面信息
page_info = await self.get_page(page_id)
if not page_info:
return None
# 构建节点
node = {
'id': page_info.get('id', ''),
'title': page_info.get('title', ''),
'url': f"{self.base_url}{page_info.get('_links', {}).get('webui', '')}",
'type': page_info.get('type', ''),
'children': []
}
# 递归获取子页面
children = await self.get_children(page_id)
for child in children:
child_id = child.get('id', '')
child_node = await self.get_page_tree(child_id, max_depth - 1)
if child_node:
node['children'].append(child_node)
return node
# 全局客户端实例(用于上下文管理)
from contextlib import asynccontextmanager
@asynccontextmanager
async def get_confluence_client():
"""获取 Confluence 客户端的异步上下文管理器"""
client = ConfluenceClient()
try:
yield client
finally:
await client.close()

355
confluence/parser.py Normal file
View File

@@ -0,0 +1,355 @@
import re
from datetime import date, datetime
from typing import Dict, Any, Optional, List
from bs4 import BeautifulSoup
import pandas as pd
from io import StringIO
from models.schemas import ShipData, PageNode, MonthlyStats
class DataParser:
"""Confluence 页面数据解析器"""
@staticmethod
def parse_ship_title(title: str) -> Optional[Dict[str, str]]:
"""
解析船舶报告标题
格式示例FZ 361#实船报告2026.02.01
Returns:
包含 location, ship_number, report_date 的字典
"""
# 匹配模式:地点 船次 #实船报告YYYY.MM.DD允许数字和#之间、#和实船报告之间有空格)
pattern = r'([A-Z]{2})\s*(\d+)\s*#?\s*实船报告(\d{4})\.(\d{2})\.(\d{2})'
match = re.search(pattern, title)
if match:
return {
'location': match.group(1),
'ship_number': match.group(2),
'report_date': date(
int(match.group(3)),
int(match.group(4)),
int(match.group(5))
)
}
# 尝试其他可能的格式
# 格式2: FZ 361# 2026.02.01
pattern2 = r'([A-Z]{2})\s*(\d+)#?\s*(\d{4})\.(\d{2})\.(\d{2})'
match2 = re.search(pattern2, title)
if match2:
return {
'location': match2.group(1),
'ship_number': match2.group(2),
'report_date': date(
int(match2.group(3)),
int(match2.group(4)),
int(match2.group(5))
)
}
return None
@staticmethod
def parse_monthly_title(title: str) -> Optional[str]:
"""
解析月度统计标题
格式示例2026.02 实船作业统计
Returns:
月份字符串,如 "2026.02"
"""
# 匹配模式YYYY.MM 实船作业统计
pattern = r'(\d{4})\.(\d{2})\s*实船作业统计'
match = re.search(pattern, title)
if match:
return f"{match.group(1)}.{match.group(2)}"
return None
@staticmethod
def extract_table_data(html_content: str) -> Optional[pd.DataFrame]:
"""
从 Confluence 页面 HTML 中提取表格数据
Args:
html_content: 页面 HTML 内容
Returns:
DataFrame 或 None
"""
if not html_content:
return None
try:
soup = BeautifulSoup(html_content, 'html.parser')
# 查找 Confluence 表格
tables = soup.find_all('table', {'class': 'confluenceTable'})
if not tables:
# 尝试查找任何表格
tables = soup.find_all('table')
if not tables:
return None
# 使用 pandas 解析第一个表格
# 注意pd.read_html 需要 lxml 或 html5lib 解析器
table_html = str(tables[0])
dfs = pd.read_html(StringIO(table_html))
if not dfs:
return None
df = dfs[0]
# 清理列名(移除 NaN 列)
df = df.dropna(axis=1, how='all')
return df
except Exception as e:
print(f"表格解析失败: {e}")
return None
@staticmethod
def dataframe_to_dict(df: pd.DataFrame) -> Dict[str, Any]:
"""
将 DataFrame 转换为字典
处理合并单元格等特殊情况
"""
if df is None or df.empty:
return {}
result = {}
# 如果只有一行,可能是横向表格
if len(df) == 1:
# 假设第一行是标题,第二行是数据
# 或者第一列是字段名
for col in df.columns:
result[str(col)] = str(df[col].iloc[0]) if not pd.isna(df[col].iloc[0]) else ""
else:
# 纵向表格,第一列可能是字段名
# 尝试识别字段-值对
for _, row in df.iterrows():
if len(row) >= 2:
key = str(row.iloc[0]) if not pd.isna(row.iloc[0]) else ""
value = str(row.iloc[1]) if not pd.isna(row.iloc[1]) else ""
if key:
result[key] = value
return result
@staticmethod
def extract_key_metrics(raw_data: Dict[str, Any]) -> Dict[str, Any]:
"""
从原始表格数据中提取关键业务指标
Returns:
包含关键指标的字典
"""
metrics = {}
# 船名
metrics['ship_name'] = raw_data.get('船名', '')
# 作业时间
metrics['operation_time'] = raw_data.get('作业时间', '')
# 作业类型
op_type = raw_data.get('作业类型(装船或卸船)', '')
metrics['operation_type'] = op_type.replace('装船', '装船').replace('卸船', '卸船') if op_type else None
# 作业箱量 (TEU) - 尝试提取数字
teu_str = raw_data.get('作业箱量 (TEU)', '')
metrics['teu'] = DataParser._extract_number(teu_str)
# 作业循环 (move)
moves_str = raw_data.get('作业循环 (move)', '')
metrics['moves'] = DataParser._extract_number(moves_str)
# 作业毛效率
gross_eff_str = raw_data.get('作业毛效率 (move/车/小时)', '')
metrics['gross_efficiency'] = DataParser._extract_number(gross_eff_str)
# 作业净效率
net_eff_str = raw_data.get('作业净效率 (move/车/小时)', '')
metrics['net_efficiency'] = DataParser._extract_number(net_eff_str)
# 故障次数
fault_count_str = raw_data.get('故障次数', '')
metrics['fault_count'] = DataParser._extract_int(fault_count_str)
# 故障率
fault_rate_str = raw_data.get('故障率', '')
metrics['fault_rate'] = DataParser._extract_percentage(fault_rate_str)
# 人工介入次数
manual_count_str = raw_data.get('人工介入次数', '')
metrics['manual_intervention_count'] = DataParser._extract_int(manual_count_str)
# 人工介入率
manual_rate_str = raw_data.get('人工介入率', '')
metrics['manual_intervention_rate'] = DataParser._extract_percentage(manual_rate_str)
return metrics
@staticmethod
def _extract_number(value_str: str) -> Optional[float]:
"""从字符串中提取数字(优先提取总和值)"""
if not value_str:
return None
try:
import re
val_str = str(value_str)
# 模式1: 提取"XX总和"或"XX(总和)"中的数字
sum_match = re.search(r'(\d+(?:\.\d+)?)\s*[(]总和[)]', val_str)
if sum_match:
return float(sum_match.group(1))
# 模式2: 提取最后一个数字(通常是总和)
# 例如: "23卸船+ 6装船 29总和" 提取 29
all_numbers = re.findall(r'(\d+(?:\.\d+)?)', val_str)
if all_numbers:
# 返回最后一个数字(通常是总和)
return float(all_numbers[-1])
except:
pass
return None
@staticmethod
def _extract_int(value_str: str) -> Optional[int]:
"""从字符串中提取整数"""
if not value_str:
return None
try:
cleaned = ''.join(c for c in str(value_str) if c.isdigit())
if cleaned:
return int(cleaned)
except:
pass
return None
@staticmethod
def _extract_percentage(value_str: str) -> Optional[float]:
"""从百分比字符串中提取数值"""
if not value_str:
return None
try:
# 移除 % 符号
cleaned = str(value_str).replace('%', '').strip()
if cleaned:
return float(cleaned)
except:
pass
return None
@staticmethod
def build_page_node(
page_id: str,
title: str,
url: str,
html_content: Optional[str] = None,
children: Optional[List[PageNode]] = None
) -> PageNode:
"""
构建页面节点
自动识别页面类型(月度统计或船舶报告)并解析数据
"""
node = PageNode(
id=page_id,
title=title,
url=url,
children=children or []
)
# 尝试解析为月度统计页面
month = DataParser.parse_monthly_title(title)
if month:
node.is_monthly = True
node.month = month
# 尝试解析为船舶报告
ship_info = DataParser.parse_ship_title(title)
if ship_info:
# 提取表格数据
raw_data = {}
if html_content:
df = DataParser.extract_table_data(html_content)
if df is not None:
raw_data = DataParser.dataframe_to_dict(df)
# 提取关键业务指标
metrics = DataParser.extract_key_metrics(raw_data)
node.data = ShipData(
ship_code=f"{ship_info['location']} {ship_info['ship_number']}#",
location=ship_info['location'],
ship_number=ship_info['ship_number'],
report_date=ship_info['report_date'],
raw_data=raw_data,
**metrics
)
return node
@staticmethod
def build_monthly_stats(node: PageNode) -> Optional[MonthlyStats]:
"""
从月度统计页面节点构建月度统计数据
遍历所有子页面(船舶报告)
"""
if not node.is_monthly or not node.month:
return None
ships = []
for child in node.children:
if child.data:
ships.append(child.data)
# 计算汇总
summary = {
'total_ships': len(ships),
'date_range': {
'start': min([s.report_date for s in ships if s.report_date], default=None),
'end': max([s.report_date for s in ships if s.report_date], default=None)
}
}
# 统计各字段汇总(如果有数值字段)
for ship in ships:
for key, value in ship.raw_data.items():
# 尝试提取数值
try:
# 移除单位,尝试解析数字
numeric_str = re.sub(r'[^\d.]', '', str(value))
if numeric_str:
num = float(numeric_str)
if key not in summary:
summary[key] = {'total': 0, 'count': 0, 'avg': 0}
summary[key]['total'] += num
summary[key]['count'] += 1
summary[key]['avg'] = summary[key]['total'] / summary[key]['count']
except:
pass
return MonthlyStats(
month=node.month,
page_id=node.id,
title=node.title,
url=node.url,
total_ships=len(ships),
ships=ships,
summary=summary
)

108
diagnose_2025.py Normal file
View File

@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""
诊断脚本:检查 2025.06 页面的子页面结构
"""
import asyncio
import httpx
from config import settings
async def check_page_children(page_id: str):
"""检查页面的直接子页面"""
base_url = settings.CONFLUENCE_BASE_URL.rstrip('/')
token = settings.CONFLUENCE_TOKEN
headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
"Accept": "application/json"
}
async with httpx.AsyncClient(headers=headers, timeout=30.0) as client:
# 1. 获取页面基本信息
url = f"{base_url}/rest/api/content/{page_id}"
print(f"\n=== 页面 {page_id} 基本信息 ===")
try:
response = await client.get(url)
response.raise_for_status()
data = response.json()
print(f"标题: {data.get('title', 'N/A')}")
print(f"类型: {data.get('type', 'N/A')}")
print(f"状态: {data.get('status', 'N/A')}")
except Exception as e:
print(f"获取页面信息失败: {e}")
return
# 2. 获取直接子页面
url = f"{base_url}/rest/api/content/{page_id}/child/page"
params = {
"limit": 100,
"expand": "_links"
}
print(f"\n=== 页面 {page_id} 的直接子页面 ===")
try:
response = await client.get(url, params=params)
response.raise_for_status()
data = response.json()
results = data.get('results', [])
size = data.get('size', 0)
print(f"子页面总数: {size}")
print(f"返回的子页面数: {len(results)}")
if results:
print("\n子页面列表:")
for i, child in enumerate(results, 1):
child_id = child.get('id', 'N/A')
title = child.get('title', 'N/A')
child_type = child.get('type', 'N/A')
print(f" {i}. ID: {child_id}, 标题: {title}, 类型: {child_type}")
# 递归检查每个子页面的子页面
await check_grandchildren(client, base_url, child_id, depth=1)
else:
print(" 该页面没有子页面")
except Exception as e:
print(f"获取子页面失败: {e}")
async def check_grandchildren(client, base_url, page_id, depth=1):
"""递归检查孙页面"""
url = f"{base_url}/rest/api/content/{page_id}/child/page"
params = {"limit": 100}
try:
response = await client.get(url, params=params)
response.raise_for_status()
data = response.json()
results = data.get('results', [])
if results:
indent = " " * (depth + 1)
print(f"{indent}└─ 子页面数: {len(results)}")
for i, child in enumerate(results, 1):
child_id = child.get('id', 'N/A')
title = child.get('title', 'N/A')
print(f"{indent} {i}. ID: {child_id}, 标题: {title}")
# 检查是否还有更多嵌套
if depth < 2:
await check_grandchildren(client, base_url, child_id, depth + 1)
except Exception as e:
indent = " " * (depth + 1)
print(f"{indent}└─ 获取失败: {e}")
async def main():
"""主函数"""
# 检查 2025.06 页面 (ID: 137446576)
page_id = "137446576"
print(f"开始检查页面 {page_id} (2025.06) 的结构...")
await check_page_children(page_id)
if __name__ == "__main__":
asyncio.run(main())

196
export_data.py Normal file
View File

@@ -0,0 +1,196 @@
#!/usr/bin/env python3
"""
船舶数据导出脚本
用法:
python export_data.py --format json --output ship_data.json
python export_data.py --format csv --output ship_data.csv
python export_data.py --analysis # 打印数据分析报告
"""
import argparse
import json
import csv
import asyncio
import sys
from datetime import datetime
from pathlib import Path
# 添加项目根目录到路径
sys.path.insert(0, str(Path(__file__).parent))
from main import get_table_data, get_data_analysis
from services.cache import cache
async def export_to_json(output_path: str):
"""导出数据到JSON文件"""
print("正在获取表格数据...")
table_data = await get_table_data()
data_dict = {
"export_time": datetime.now().isoformat(),
"total_records": table_data.total_records,
"fields": table_data.fields,
"records_with_data": table_data.records_with_data,
"records_without_data": table_data.records_without_data,
"data": [row.model_dump() for row in table_data.data]
}
with open(output_path, 'w', encoding='utf-8') as f:
json.dump(data_dict, f, ensure_ascii=False, indent=2)
print(f"✓ 数据已导出到: {output_path}")
print(f" - 总记录数: {table_data.total_records}")
print(f" - 有业务数据: {table_data.records_with_data}")
print(f" - 无业务数据: {table_data.records_without_data}")
async def export_to_csv(output_path: str):
"""导出数据到CSV文件"""
print("正在获取表格数据...")
table_data = await get_table_data()
with open(output_path, 'w', newline='', encoding='utf-8-sig') as f:
writer = csv.writer(f)
# 写入表头
headers = [
"船舶代码", "船名", "报告日期", "月份", "作业时间", "作业类型",
"TEU", "Moves", "毛效率", "净效率",
"故障次数", "故障率", "人工介入次数", "人工介入率",
"页面ID", "页面URL", "是否有完整数据"
]
writer.writerow(headers)
# 写入数据行
for row in table_data.data:
writer.writerow([
row.ship_code,
row.ship_name or "",
row.report_date or "",
row.month or "",
row.operation_time or "",
row.operation_type or "",
row.teu if row.teu is not None else "",
row.moves if row.moves is not None else "",
row.gross_efficiency if row.gross_efficiency is not None else "",
row.net_efficiency if row.net_efficiency is not None else "",
row.fault_count if row.fault_count is not None else "",
row.fault_rate if row.fault_rate is not None else "",
row.manual_intervention_count if row.manual_intervention_count is not None else "",
row.manual_intervention_rate if row.manual_intervention_rate is not None else "",
row.page_id,
row.page_url,
"" if row.has_complete_data else ""
])
print(f"✓ 数据已导出到: {output_path}")
print(f" - 总记录数: {table_data.total_records}")
print(f" - 有业务数据: {table_data.records_with_data}")
print(f" - 无业务数据: {table_data.records_without_data}")
async def print_analysis():
"""打印数据分析报告"""
print("正在生成数据分析报告...\n")
analysis = await get_data_analysis()
print("=" * 60)
print("船舶数据分析报告")
print("=" * 60)
print(f"\n📊 总体统计:")
print(f" - 船舶总数: {analysis.total_ships}")
print(f" - 有TEU数据: {analysis.ships_with_teu} ({analysis.ships_with_teu/analysis.total_ships*100:.1f}%)")
print(f" - 有Moves数据: {analysis.ships_with_moves} ({analysis.ships_with_moves/analysis.total_ships*100:.1f}%)")
print(f" - 有效率数据: {analysis.ships_with_efficiency} ({analysis.ships_with_efficiency/analysis.total_ships*100:.1f}%)")
print(f" - 有故障数据: {analysis.ships_with_faults} ({analysis.ships_with_faults/analysis.total_ships*100:.1f}%)")
print(f" - 有人工介入数据: {analysis.ships_with_manual} ({analysis.ships_with_manual/analysis.total_ships*100:.1f}%)")
print(f"\n📅 月份分布:")
for month, count in sorted(analysis.monthly_distribution.items()):
print(f" - {month}: {count} 条记录")
print(f"\n⚓ 作业类型分布:")
for op_type, count in analysis.operation_type_distribution.items():
print(f" - {op_type}: {count} 条记录")
if analysis.teu_stats.get('total'):
print(f"\n📦 TEU统计:")
print(f" - 最小值: {analysis.teu_stats['min']:.2f}")
print(f" - 最大值: {analysis.teu_stats['max']:.2f}")
print(f" - 平均值: {analysis.teu_stats['avg']:.2f}")
print(f" - 总计: {analysis.teu_stats['total']:.2f}")
if analysis.moves_stats.get('total'):
print(f"\n🔄 Moves统计:")
print(f" - 最小值: {analysis.moves_stats['min']:.2f}")
print(f" - 最大值: {analysis.moves_stats['max']:.2f}")
print(f" - 平均值: {analysis.moves_stats['avg']:.2f}")
print(f" - 总计: {analysis.moves_stats['total']:.2f}")
if analysis.efficiency_stats.get('avg'):
print(f"\n⚡ 效率统计:")
print(f" - 最小值: {analysis.efficiency_stats['min']:.2f}")
print(f" - 最大值: {analysis.efficiency_stats['max']:.2f}")
print(f" - 平均值: {analysis.efficiency_stats['avg']:.2f}")
print("\n" + "=" * 60)
print("可用字段列表:")
print("=" * 60)
fields = [
("ship_code", "船舶代码"),
("ship_name", "船名"),
("report_date", "报告日期"),
("month", "所属月份"),
("operation_time", "作业时间"),
("operation_type", "作业类型"),
("teu", "作业箱量 (TEU)"),
("moves", "作业循环 (move)"),
("gross_efficiency", "作业毛效率 (move/车/小时)"),
("net_efficiency", "作业净效率 (move/车/小时)"),
("fault_count", "故障次数"),
("fault_rate", "故障率 (%)"),
("manual_intervention_count", "人工介入次数"),
("manual_intervention_rate", "人工介入率 (%)"),
("page_id", "Confluence页面ID"),
("page_url", "Confluence页面URL"),
]
for field, desc in fields:
print(f" - {field}: {desc}")
print("\n")
async def main():
parser = argparse.ArgumentParser(description="船舶数据导出工具")
parser.add_argument("--format", choices=["json", "csv"], default="json",
help="导出格式 (默认: json)")
parser.add_argument("--output", type=str, default=None,
help="输出文件路径")
parser.add_argument("--analysis", action="store_true",
help="打印数据分析报告")
args = parser.parse_args()
if args.analysis:
await print_analysis()
return
# 设置默认输出文件名
if not args.output:
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
args.output = f"ship_data_{timestamp}.{args.format}"
# 执行导出
if args.format == "json":
await export_to_json(args.output)
else:
await export_to_csv(args.output)
if __name__ == "__main__":
asyncio.run(main())

103
get_independent_data.py Normal file
View File

@@ -0,0 +1,103 @@
"""补充获取独立页面的数据"""
import asyncio
import sys
sys.path.insert(0, '.')
from confluence.client import ConfluenceClient
from confluence.parser import DataParser
from models.schemas import ShipData, PageNode
# 独立页面ID列表不在页面树下的页面
INDEPENDENT_PAGES = [
"159032145", # FZ 231# 2025.12.20
"159049306", # FZ 251# 2026.01.01
# 可以添加更多独立页面ID
]
async def get_independent_ship_data():
"""获取独立页面的船舶数据"""
print("=" * 60)
print("🔍 获取独立页面数据")
print("=" * 60)
client = ConfluenceClient()
independent_ships = []
for page_id in INDEPENDENT_PAGES:
try:
print(f"\n📄 获取页面 {page_id}...")
# 获取页面信息
page_info = await client.get_page(page_id)
if not page_info:
print(f" ❌ 页面 {page_id} 不存在")
continue
title = page_info.get('title', '')
print(f" 标题: {title}")
# 解析船舶标题
ship_info = DataParser.parse_ship_title(title)
if not ship_info:
print(f" ⚠️ 无法解析标题: {title}")
continue
# 获取页面内容
html_content = await client.get_page_content(page_id)
if not html_content:
print(f" ❌ 无法获取内容")
continue
# 提取表格数据
df = DataParser.extract_table_data(html_content)
raw_data = {}
if df is not None:
raw_data = DataParser.dataframe_to_dict(df)
# 提取关键指标
metrics = DataParser.extract_key_metrics(raw_data)
# 创建ShipData
ship_data = ShipData(
ship_code=f"{ship_info['location']} {ship_info['ship_number']}#",
location=ship_info['location'],
ship_number=ship_info['ship_number'],
report_date=ship_info['report_date'],
raw_data=raw_data,
**metrics
)
# 确定月份
if ship_data.report_date:
month = f"{ship_data.report_date.year}.{ship_data.report_date.month:02d}"
else:
month = None
independent_ships.append({
'page_id': page_id,
'title': title,
'ship_data': ship_data,
'month': month
})
print(f" ✅ 成功: {ship_data.ship_code} - {ship_data.report_date} - TEU: {ship_data.teu}")
except Exception as e:
print(f" ❌ 错误: {e}")
await client.close()
print(f"\n{'=' * 60}")
print(f"✅ 获取到 {len(independent_ships)} 条独立数据")
print(f"{'=' * 60}")
return independent_ships
if __name__ == "__main__":
ships = asyncio.run(get_independent_ship_data())
# 打印汇总
print("\n📊 独立数据汇总:")
for ship in ships:
data = ship['ship_data']
print(f" {data.ship_code} ({ship['month']}): TEU={data.teu}, Moves={data.moves}")

794
main.py Normal file
View File

@@ -0,0 +1,794 @@
from fastapi import FastAPI, HTTPException, Depends
from fastapi.staticfiles import StaticFiles
from fastapi.responses import HTMLResponse
from fastapi.middleware.cors import CORSMiddleware
from typing import List, Dict, Any, Optional
import os
from contextlib import asynccontextmanager
from confluence.client import ConfluenceClient, get_confluence_client
from confluence.parser import DataParser
from models.schemas import (
PageNode, PageTreeResponse, MonthlyStats, SummaryStatistics, ShipData,
TableDataResponse, TableDataRow, DataAnalysisResponse,
AnalysisSummaryResponse, MonthlySummary,
DailyStats, DailyStatsResponse, DailyStatsShip
)
from services.cache import cache
from config import settings
# 应用生命周期管理
@asynccontextmanager
async def lifespan(app: FastAPI):
"""应用启动和关闭时的处理"""
# 启动时清理过期缓存
cleared = cache.clear_expired()
if cleared > 0:
print(f"已清理 {cleared} 个过期缓存文件")
yield
# 关闭时的清理(如果需要)
pass
# 创建 FastAPI 应用
app = FastAPI(
title="福州江阴实船作业统计可视化系统",
description="Confluence 数据采集与可视化报表",
version="1.0.0",
lifespan=lifespan
)
# CORS 配置
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# 静态文件服务
if os.path.exists("static"):
app.mount("/static", StaticFiles(directory="static"), name="static")
@app.get("/", response_class=HTMLResponse)
async def root():
"""首页 - 重定向到可视化页面"""
return """
<!DOCTYPE html>
<html>
<head>
<title>福州江阴实船作业统计</title>
<meta http-equiv="refresh" content="0;url=/static/index.html">
</head>
<body>
<p>正在跳转到可视化页面...</p>
<p>如果没有自动跳转,请<a href="/static/index.html">点击这里</a></p>
</body>
</html>
"""
@app.get("/api/health")
async def health_check():
"""健康检查"""
return {
"status": "healthy",
"version": "1.0.0",
"confluence_configured": bool(settings.CONFLUENCE_TOKEN)
}
@app.get("/api/pages/tree", response_model=PageTreeResponse)
async def get_page_tree():
"""
获取完整页面树
从根页面开始,递归获取所有月度统计页面和船舶报告页面
"""
# 检查缓存
cached = cache.get("page_tree")
if cached:
return PageTreeResponse(**cached)
async with get_confluence_client() as client:
# 获取页面树
tree_data = await client.get_page_tree(settings.ROOT_PAGE_ID, max_depth=4)
if not tree_data:
raise HTTPException(status_code=404, detail="无法获取页面树")
# 转换为 PageNode 并补充数据
root_node = await _build_page_node_recursive(client, tree_data)
# 统计信息
total_months = 0
total_reports = 0
def count_stats(node: PageNode):
nonlocal total_months, total_reports
if node.is_monthly:
total_months += 1
if node.data:
total_reports += 1
for child in node.children:
count_stats(child)
count_stats(root_node)
response = PageTreeResponse(
root_page=root_node,
total_months=total_months,
total_ship_reports=total_reports
)
# 缓存结果
cache.set("page_tree", response.model_dump())
return response
async def _build_page_node_recursive(
client: ConfluenceClient,
tree_data: Dict[str, Any]
) -> PageNode:
"""递归构建页面节点"""
page_id = tree_data.get('id', '')
title = tree_data.get('title', '')
url = tree_data.get('url', '')
# 获取页面内容(如果是船舶报告页面)
html_content = None
ship_info = DataParser.parse_ship_title(title)
if ship_info:
html_content = await client.get_page_content(page_id)
# 构建当前节点
node = DataParser.build_page_node(
page_id=page_id,
title=title,
url=url,
html_content=html_content
)
# 递归处理子页面
for child_data in tree_data.get('children', []):
child_node = await _build_page_node_recursive(client, child_data)
node.children.append(child_node)
return node
@app.get("/api/pages/{page_id}")
async def get_page_detail(page_id: str):
"""
获取指定页面的详细数据
包含解析后的船舶报告数据(如果是船舶报告页面)
"""
cache_key = f"page_{page_id}"
cached = cache.get(cache_key)
if cached:
return cached
async with get_confluence_client() as client:
# 获取页面信息
page_info = await client.get_page(page_id)
if not page_info:
raise HTTPException(status_code=404, detail=f"页面 {page_id} 不存在")
title = page_info.get('title', '')
url = f"{settings.CONFLUENCE_BASE_URL}{page_info.get('_links', {}).get('webui', '')}"
# 获取页面内容
html_content = await client.get_page_content(page_id)
# 构建节点
node = DataParser.build_page_node(
page_id=page_id,
title=title,
url=url,
html_content=html_content
)
result = {
"page": node.model_dump(),
"raw_html": html_content[:1000] if html_content else None # 限制返回大小
}
cache.set(cache_key, result)
return result
@app.get("/api/monthly/{month}")
async def get_monthly_stats(month: str):
"""
获取指定月份的统计数据
month 格式: YYYY.MM (如 2026.02)
"""
cache_key = f"monthly_{month}"
cached = cache.get(cache_key)
if cached:
return cached
# 先获取完整页面树
tree_response = await get_page_tree()
# 查找对应月份的节点
def find_monthly_node(node: PageNode) -> Optional[PageNode]:
if node.is_monthly and node.month == month:
return node
for child in node.children:
found = find_monthly_node(child)
if found:
return found
return None
monthly_node = find_monthly_node(tree_response.root_page)
if not monthly_node:
raise HTTPException(status_code=404, detail=f"未找到月份 {month} 的数据")
# 获取子页面详细信息
async with get_confluence_client() as client:
for i, child in enumerate(monthly_node.children):
if not child.data:
html_content = await client.get_page_content(child.id)
if html_content:
child_node = DataParser.build_page_node(
page_id=child.id,
title=child.title,
url=child.url,
html_content=html_content
)
monthly_node.children[i] = child_node
# 构建月度统计
stats = DataParser.build_monthly_stats(monthly_node)
if not stats:
raise HTTPException(status_code=500, detail="构建月度统计数据失败")
result = stats.model_dump()
cache.set(cache_key, result)
return result
@app.get("/api/statistics/summary")
async def get_summary_statistics():
"""
获取整体汇总统计
包含所有月份的趋势、船次频率等信息
"""
cache_key = "summary_stats"
cached = cache.get(cache_key)
if cached:
return SummaryStatistics(**cached)
# 获取页面树
tree_response = await get_page_tree()
# 收集所有月度数据
monthly_stats_list = []
ship_frequency = {}
all_ships = []
def collect_monthly_data(node: PageNode):
if node.is_monthly:
stats = DataParser.build_monthly_stats(node)
if stats:
monthly_stats_list.append(stats)
# 统计船次频率
for ship in stats.ships:
ship_code = ship.ship_code
ship_frequency[ship_code] = ship_frequency.get(ship_code, 0) + 1
all_ships.append(ship)
for child in node.children:
collect_monthly_data(child)
collect_monthly_data(tree_response.root_page)
# 按月份排序
monthly_stats_list.sort(key=lambda x: x.month)
# 构建月度趋势数据
monthly_trend = []
for stats in monthly_stats_list:
monthly_trend.append({
"month": stats.month,
"total_ships": stats.total_ships,
"summary": stats.summary
})
# 计算日期范围
report_dates = [s.report_date for s in all_ships if s.report_date]
date_range = {
"start": min(report_dates).isoformat() if report_dates else None,
"end": max(report_dates).isoformat() if report_dates else None
}
summary = SummaryStatistics(
total_months=len(monthly_stats_list),
total_ship_reports=len(all_ships),
date_range=date_range,
monthly_trend=monthly_trend,
ship_frequency=ship_frequency,
monthly_data=monthly_stats_list
)
cache.set(cache_key, summary.model_dump())
return summary
@app.post("/api/refresh")
async def refresh_cache():
"""
强制刷新所有缓存
重新从 Confluence 获取最新数据
"""
cache.clear()
return {"message": "缓存已清除", "timestamp": datetime.now().isoformat()}
@app.get("/api/analysis/table-data", response_model=TableDataResponse)
async def get_table_data():
"""
获取所有船舶的表格数据
返回包含关键业务字段的完整表格数据
"""
cache_key = "table_data"
cached = cache.get(cache_key)
if cached:
return TableDataResponse(**cached)
# 获取页面树
tree_response = await get_page_tree()
# 收集所有船舶数据
all_ships = []
def collect_ships(node: PageNode, month: Optional[str] = None):
if node.is_monthly:
month = node.month
if node.data:
all_ships.append((node, month))
for child in node.children:
collect_ships(child, month)
collect_ships(tree_response.root_page)
# 构建表格数据行
table_rows = []
for node, month in all_ships:
ship = node.data
has_data = any([
ship.teu is not None,
ship.moves is not None,
ship.gross_efficiency is not None,
ship.net_efficiency is not None
])
row = TableDataRow(
ship_code=ship.ship_code,
ship_name=ship.ship_name,
report_date=ship.report_date.isoformat() if ship.report_date else None,
month=month,
operation_time=ship.operation_time,
operation_type=ship.operation_type,
teu=ship.teu,
moves=ship.moves,
gross_efficiency=ship.gross_efficiency,
net_efficiency=ship.net_efficiency,
fault_count=ship.fault_count,
fault_rate=ship.fault_rate,
manual_intervention_count=ship.manual_intervention_count,
manual_intervention_rate=ship.manual_intervention_rate,
page_id=node.id,
page_url=node.url,
has_complete_data=has_data
)
table_rows.append(row)
# 按报告日期排序
table_rows.sort(key=lambda x: x.report_date or "")
# 统计
records_with_data = sum(1 for r in table_rows if r.has_complete_data)
response = TableDataResponse(
total_records=len(table_rows),
fields=[
"ship_code", "ship_name", "report_date", "month",
"operation_time", "operation_type", "teu", "moves",
"gross_efficiency", "net_efficiency",
"fault_count", "fault_rate",
"manual_intervention_count", "manual_intervention_rate",
"page_id", "page_url", "has_complete_data"
],
records_with_data=records_with_data,
records_without_data=len(table_rows) - records_with_data,
data=table_rows
)
cache.set(cache_key, response.model_dump())
return response
@app.get("/api/analysis/table-data/analysis", response_model=DataAnalysisResponse)
async def get_data_analysis():
"""
获取表格数据的统计分析
展示哪些船次有作业箱量、效率等数据
"""
cache_key = "data_analysis"
cached = cache.get(cache_key)
if cached:
return DataAnalysisResponse(**cached)
# 获取表格数据
table_data = await get_table_data()
rows = table_data.data
# 统计有数据的船次
ships_with_teu = sum(1 for r in rows if r.teu is not None)
ships_with_moves = sum(1 for r in rows if r.moves is not None)
ships_with_efficiency = sum(1 for r in rows if r.gross_efficiency is not None or r.net_efficiency is not None)
ships_with_faults = sum(1 for r in rows if r.fault_count is not None)
ships_with_manual = sum(1 for r in rows if r.manual_intervention_count is not None)
# 月份分布
monthly_distribution = {}
for r in rows:
month = r.month or "未知"
monthly_distribution[month] = monthly_distribution.get(month, 0) + 1
# 作业类型分布
operation_type_distribution = {}
for r in rows:
op_type = r.operation_type or "未知"
operation_type_distribution[op_type] = operation_type_distribution.get(op_type, 0) + 1
# TEU统计
teu_values = [r.teu for r in rows if r.teu is not None]
teu_stats = {
"min": min(teu_values) if teu_values else None,
"max": max(teu_values) if teu_values else None,
"avg": sum(teu_values) / len(teu_values) if teu_values else None,
"total": sum(teu_values) if teu_values else None
}
# Moves统计
moves_values = [r.moves for r in rows if r.moves is not None]
moves_stats = {
"min": min(moves_values) if moves_values else None,
"max": max(moves_values) if moves_values else None,
"avg": sum(moves_values) / len(moves_values) if moves_values else None,
"total": sum(moves_values) if moves_values else None
}
# 效率统计
efficiency_values = [r.gross_efficiency for r in rows if r.gross_efficiency is not None]
efficiency_stats = {
"min": min(efficiency_values) if efficiency_values else None,
"max": max(efficiency_values) if efficiency_values else None,
"avg": sum(efficiency_values) / len(efficiency_values) if efficiency_values else None
}
response = DataAnalysisResponse(
total_ships=len(rows),
ships_with_teu=ships_with_teu,
ships_with_moves=ships_with_moves,
ships_with_efficiency=ships_with_efficiency,
ships_with_faults=ships_with_faults,
ships_with_manual=ships_with_manual,
monthly_distribution=monthly_distribution,
operation_type_distribution=operation_type_distribution,
teu_stats=teu_stats,
moves_stats=moves_stats,
efficiency_stats=efficiency_stats
)
cache.set(cache_key, response.model_dump())
return response
@app.get("/api/analysis/table-data/export")
async def export_table_data(format: str = "json"):
"""
导出表格数据
支持格式: json, csv
"""
from fastapi.responses import PlainTextResponse
import json
import csv
from io import StringIO
# 获取表格数据
table_data = await get_table_data()
if format.lower() == "csv":
# 生成CSV
output = StringIO()
writer = csv.writer(output)
# 写入表头
headers = [
"船舶代码", "船名", "报告日期", "月份", "作业时间", "作业类型",
"TEU", "Moves", "毛效率", "净效率",
"故障次数", "故障率", "人工介入次数", "人工介入率",
"页面ID", "页面URL"
]
writer.writerow(headers)
# 写入数据行
for row in table_data.data:
writer.writerow([
row.ship_code,
row.ship_name or "",
row.report_date or "",
row.month or "",
row.operation_time or "",
row.operation_type or "",
row.teu or "",
row.moves or "",
row.gross_efficiency or "",
row.net_efficiency or "",
row.fault_count or "",
row.fault_rate or "",
row.manual_intervention_count or "",
row.manual_intervention_rate or "",
row.page_id,
row.page_url
])
csv_content = output.getvalue()
output.close()
return PlainTextResponse(
content=csv_content,
media_type="text/csv",
headers={"Content-Disposition": "attachment; filename=ship_data.csv"}
)
else: # JSON格式
# 转换为可序列化的字典
data_dict = {
"total_records": table_data.total_records,
"fields": table_data.fields,
"records_with_data": table_data.records_with_data,
"records_without_data": table_data.records_without_data,
"data": [row.model_dump() for row in table_data.data]
}
json_content = json.dumps(data_dict, ensure_ascii=False, indent=2)
return PlainTextResponse(
content=json_content,
media_type="application/json",
headers={"Content-Disposition": "attachment; filename=ship_data.json"}
)
from datetime import datetime
@app.get("/api/analysis/summary", response_model=AnalysisSummaryResponse)
async def get_analysis_summary():
"""
获取数据分析摘要
统计各月份作业情况、作业类型分布、平均TEU和效率等
"""
cache_key = "analysis_summary"
cached = cache.get(cache_key)
if cached:
return AnalysisSummaryResponse(**cached)
# 获取表格数据
table_data = await get_table_data()
rows = table_data.data
if not rows:
return AnalysisSummaryResponse(
total_records=0,
monthly_summary=[],
operation_type_distribution={},
date_range={},
overall_avg_teu=None,
overall_avg_efficiency=None,
total_teu=None,
total_moves=None
)
# 按月份分组统计
monthly_stats: Dict[str, Dict] = {}
operation_type_count: Dict[str, int] = {}
all_teu_values = []
all_efficiency_values = []
all_moves_values = []
for row in rows:
month = row.month or "未知"
# 初始化月份统计
if month not in monthly_stats:
monthly_stats[month] = {
"total_ships": 0,
"loading_count": 0,
"unloading_count": 0,
"teu_values": [],
"moves_values": [],
"gross_efficiency_values": [],
"net_efficiency_values": []
}
stats = monthly_stats[month]
stats["total_ships"] += 1
# 作业类型统计
op_type = row.operation_type
if op_type:
operation_type_count[op_type] = operation_type_count.get(op_type, 0) + 1
if "装船" in op_type:
stats["loading_count"] += 1
elif "卸船" in op_type:
stats["unloading_count"] += 1
# 收集数值
if row.teu is not None:
stats["teu_values"].append(row.teu)
all_teu_values.append(row.teu)
if row.moves is not None:
stats["moves_values"].append(row.moves)
all_moves_values.append(row.moves)
if row.gross_efficiency is not None:
stats["gross_efficiency_values"].append(row.gross_efficiency)
all_efficiency_values.append(row.gross_efficiency)
if row.net_efficiency is not None:
stats["net_efficiency_values"].append(row.net_efficiency)
# 构建月度汇总
monthly_summary_list = []
for month, stats in sorted(monthly_stats.items()):
teu_values = stats["teu_values"]
moves_values = stats["moves_values"]
gross_eff_values = stats["gross_efficiency_values"]
net_eff_values = stats["net_efficiency_values"]
monthly_summary = MonthlySummary(
month=month,
total_ships=stats["total_ships"],
loading_count=stats["loading_count"],
unloading_count=stats["unloading_count"],
avg_teu=sum(teu_values) / len(teu_values) if teu_values else None,
total_teu=sum(teu_values) if teu_values else None,
avg_moves=sum(moves_values) / len(moves_values) if moves_values else None,
total_moves=sum(moves_values) if moves_values else None,
avg_gross_efficiency=sum(gross_eff_values) / len(gross_eff_values) if gross_eff_values else None,
avg_net_efficiency=sum(net_eff_values) / len(net_eff_values) if net_eff_values else None
)
monthly_summary_list.append(monthly_summary)
# 计算整体统计
date_range = {
"start": min([r.report_date for r in rows if r.report_date], default=None),
"end": max([r.report_date for r in rows if r.report_date], default=None)
}
response = AnalysisSummaryResponse(
total_records=len(rows),
date_range=date_range,
monthly_summary=monthly_summary_list,
operation_type_distribution=operation_type_count,
overall_avg_teu=sum(all_teu_values) / len(all_teu_values) if all_teu_values else None,
overall_avg_efficiency=sum(all_efficiency_values) / len(all_efficiency_values) if all_efficiency_values else None,
total_teu=sum(all_teu_values) if all_teu_values else None,
total_moves=sum(all_moves_values) if all_moves_values else None
)
cache.set(cache_key, response.model_dump())
return response
@app.get("/api/analysis/daily-stats", response_model=DailyStatsResponse)
async def get_daily_stats():
"""
获取按日期的统计数据
按 report_date 分组统计每天的船次数量、TEU总量、Moves总量和平均效率
"""
cache_key = "daily_stats"
cached = cache.get(cache_key)
if cached:
return DailyStatsResponse(**cached)
# 获取表格数据
table_data = await get_table_data()
rows = table_data.data
# 按日期分组
daily_data: Dict[str, List[TableDataRow]] = {}
for row in rows:
date_str = row.report_date
if not date_str:
continue
if date_str not in daily_data:
daily_data[date_str] = []
daily_data[date_str].append(row)
# 构建每日统计
daily_stats_list = []
for date_str, ships in sorted(daily_data.items()):
total_teu = 0.0
total_moves = 0.0
efficiency_values = []
ships_detail = []
for row in ships:
if row.teu is not None:
total_teu += row.teu
if row.moves is not None:
total_moves += row.moves
if row.gross_efficiency is not None:
efficiency_values.append(row.gross_efficiency)
ship_detail = DailyStatsShip(
ship_code=row.ship_code,
ship_name=row.ship_name,
operation_time=row.operation_time,
operation_type=row.operation_type,
teu=row.teu,
moves=row.moves,
gross_efficiency=row.gross_efficiency,
net_efficiency=row.net_efficiency,
page_id=row.page_id,
page_url=row.page_url
)
ships_detail.append(ship_detail)
daily_stat = DailyStats(
date=date_str,
ship_count=len(ships),
total_teu=total_teu,
total_moves=total_moves,
avg_efficiency=sum(efficiency_values) / len(efficiency_values) if efficiency_values else None,
ships=ships_detail
)
daily_stats_list.append(daily_stat)
# 计算汇总统计
total_days = len(daily_stats_list)
max_ships_per_day = max([d.ship_count for d in daily_stats_list]) if daily_stats_list else 0
max_teu_per_day = max([d.total_teu for d in daily_stats_list]) if daily_stats_list else None
response = DailyStatsResponse(
daily_stats=daily_stats_list,
total_days=total_days,
max_ships_per_day=max_ships_per_day,
max_teu_per_day=max_teu_per_day
)
cache.set(cache_key, response.model_dump())
return response
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host=settings.APP_HOST, port=settings.APP_PORT)

0
models/__init__.py Normal file
View File

168
models/schemas.py Normal file
View File

@@ -0,0 +1,168 @@
from pydantic import BaseModel, Field
from typing import List, Optional, Dict, Any
from datetime import date
class ShipData(BaseModel):
"""船舶报告数据模型"""
ship_code: str = Field(..., description="船舶代码,如 FZ 361#")
location: str = Field(..., description="地点代码,如 FZ=福州")
ship_number: str = Field(..., description="船次编号,如 361#")
report_date: Optional[date] = Field(None, description="报告日期")
# 关键业务字段
ship_name: Optional[str] = Field(None, description="船名")
operation_time: Optional[str] = Field(None, description="作业时间")
operation_type: Optional[str] = Field(None, description="作业类型:装船/卸船")
teu: Optional[float] = Field(None, description="作业箱量 (TEU)")
moves: Optional[float] = Field(None, description="作业循环 (move)")
gross_efficiency: Optional[float] = Field(None, description="作业毛效率 (move/车/小时)")
net_efficiency: Optional[float] = Field(None, description="作业净效率 (move/车/小时)")
fault_count: Optional[int] = Field(None, description="故障次数")
fault_rate: Optional[float] = Field(None, description="故障率 (%)")
manual_intervention_count: Optional[int] = Field(None, description="人工介入次数")
manual_intervention_rate: Optional[float] = Field(None, description="人工介入率 (%)")
raw_data: Dict[str, Any] = Field(default_factory=dict, description="原始表格数据")
class PageNode(BaseModel):
"""Confluence 页面节点"""
id: str
title: str
url: str
children: List['PageNode'] = Field(default_factory=list)
data: Optional[ShipData] = None
is_monthly: bool = Field(False, description="是否为月度统计页面")
month: Optional[str] = Field(None, description="月份,如 2026.02")
class MonthlyStats(BaseModel):
"""月度统计数据"""
month: str = Field(..., description="月份,如 2026.02")
page_id: str
title: str
url: str
total_ships: int = Field(0, description="船舶报告总数")
ships: List[ShipData] = Field(default_factory=list)
summary: Dict[str, Any] = Field(default_factory=dict, description="汇总统计")
class PageTreeResponse(BaseModel):
"""页面树响应"""
root_page: PageNode
total_months: int = Field(0, description="月度统计页面数")
total_ship_reports: int = Field(0, description="船舶报告总数")
class SummaryStatistics(BaseModel):
"""整体汇总统计"""
total_months: int
total_ship_reports: int
date_range: Dict[str, Optional[str]] = Field(default_factory=dict)
monthly_trend: List[Dict[str, Any]] = Field(default_factory=list)
ship_frequency: Dict[str, int] = Field(default_factory=dict)
monthly_data: List[MonthlyStats] = Field(default_factory=list)
class TableDataRow(BaseModel):
"""表格数据行"""
ship_code: str = Field(..., description="船舶代码")
ship_name: Optional[str] = Field(None, description="船名")
report_date: Optional[str] = Field(None, description="报告日期")
month: Optional[str] = Field(None, description="所属月份")
operation_time: Optional[str] = Field(None, description="作业时间")
operation_type: Optional[str] = Field(None, description="作业类型")
teu: Optional[float] = Field(None, description="作业箱量 (TEU)")
moves: Optional[float] = Field(None, description="作业循环 (move)")
gross_efficiency: Optional[float] = Field(None, description="作业毛效率")
net_efficiency: Optional[float] = Field(None, description="作业净效率")
fault_count: Optional[int] = Field(None, description="故障次数")
fault_rate: Optional[float] = Field(None, description="故障率 (%)")
manual_intervention_count: Optional[int] = Field(None, description="人工介入次数")
manual_intervention_rate: Optional[float] = Field(None, description="人工介入率 (%)")
page_id: str = Field(..., description="Confluence页面ID")
page_url: str = Field(..., description="Confluence页面URL")
has_complete_data: bool = Field(False, description="是否有完整数据")
class TableDataResponse(BaseModel):
"""表格数据响应"""
total_records: int = Field(..., description="总记录数")
fields: List[str] = Field(..., description="可用字段列表")
records_with_data: int = Field(0, description="有业务数据的记录数")
records_without_data: int = Field(0, description="无业务数据的记录数")
data: List[TableDataRow] = Field(..., description="表格数据")
class DataAnalysisResponse(BaseModel):
"""数据分析响应"""
total_ships: int = Field(..., description="船舶总数")
ships_with_teu: int = Field(0, description="有TEU数据的船次")
ships_with_moves: int = Field(0, description="有moves数据的船次")
ships_with_efficiency: int = Field(0, description="有效率数据的船次")
ships_with_faults: int = Field(0, description="有故障数据的船次")
ships_with_manual: int = Field(0, description="有人工介入数据的船次")
monthly_distribution: Dict[str, int] = Field(default_factory=dict, description="月份分布")
operation_type_distribution: Dict[str, int] = Field(default_factory=dict, description="作业类型分布")
teu_stats: Dict[str, Optional[float]] = Field(default_factory=dict, description="TEU统计")
moves_stats: Dict[str, Optional[float]] = Field(default_factory=dict, description="Moves统计")
efficiency_stats: Dict[str, Optional[float]] = Field(default_factory=dict, description="效率统计")
class MonthlySummary(BaseModel):
"""月度汇总统计"""
month: str = Field(..., description="月份")
total_ships: int = Field(0, description="船次数")
loading_count: int = Field(0, description="装船次数")
unloading_count: int = Field(0, description="卸船次数")
avg_teu: Optional[float] = Field(None, description="平均TEU")
total_teu: Optional[float] = Field(None, description="总TEU")
avg_moves: Optional[float] = Field(None, description="平均Moves")
total_moves: Optional[float] = Field(None, description="总Moves")
avg_gross_efficiency: Optional[float] = Field(None, description="平均毛效率")
avg_net_efficiency: Optional[float] = Field(None, description="平均净效率")
class AnalysisSummaryResponse(BaseModel):
"""分析摘要响应"""
total_records: int = Field(..., description="总记录数")
date_range: Dict[str, Optional[str]] = Field(default_factory=dict, description="数据日期范围")
monthly_summary: List[MonthlySummary] = Field(default_factory=list, description="各月度统计")
operation_type_distribution: Dict[str, int] = Field(default_factory=dict, description="作业类型分布")
overall_avg_teu: Optional[float] = Field(None, description="整体平均TEU")
overall_avg_efficiency: Optional[float] = Field(None, description="整体平均效率")
total_teu: Optional[float] = Field(None, description="总TEU")
total_moves: Optional[float] = Field(None, description="总Moves")
class DailyStatsShip(BaseModel):
"""每日统计中的船舶信息"""
ship_code: str = Field(..., description="船舶代码")
ship_name: Optional[str] = Field(None, description="船名")
operation_time: Optional[str] = Field(None, description="作业时间")
operation_type: Optional[str] = Field(None, description="作业类型")
teu: Optional[float] = Field(None, description="作业箱量 (TEU)")
moves: Optional[float] = Field(None, description="作业循环 (move)")
gross_efficiency: Optional[float] = Field(None, description="作业毛效率")
net_efficiency: Optional[float] = Field(None, description="作业净效率")
page_id: str = Field(..., description="Confluence页面ID")
page_url: str = Field(..., description="Confluence页面URL")
class DailyStats(BaseModel):
"""每日统计数据"""
date: str = Field(..., description="日期")
ship_count: int = Field(..., description="当天作业船次数量")
total_teu: float = Field(..., description="当天TEU总量")
total_moves: float = Field(..., description="当天Moves总量")
avg_efficiency: Optional[float] = Field(None, description="当天平均效率")
ships: List[DailyStatsShip] = Field(default_factory=list, description="当天所有船舶的详细列表")
class DailyStatsResponse(BaseModel):
"""每日统计响应"""
daily_stats: List[DailyStats] = Field(default_factory=list, description="每日统计列表")
total_days: int = Field(0, description="总天数")
max_ships_per_day: int = Field(0, description="单天最大船次数")
max_teu_per_day: Optional[float] = Field(None, description="单天最大TEU")

10
requirements.txt Normal file
View File

@@ -0,0 +1,10 @@
fastapi>=0.104.0
uvicorn[standard]>=0.24.0
httpx>=0.25.0
beautifulsoup4>=4.12.0
pandas>=2.0.0
lxml>=4.9.0
python-multipart>=0.0.6
python-dotenv>=1.0.0
pydantic>=2.0.0
pydantic-settings>=2.0.0

0
services/__init__.py Normal file
View File

83
services/cache.py Normal file
View File

@@ -0,0 +1,83 @@
import json
import os
from datetime import datetime, timedelta
from typing import Any, Optional
from pathlib import Path
from config import settings
class CacheManager:
"""文件缓存管理器"""
def __init__(self):
self.cache_dir = Path(settings.CACHE_DIR)
self.cache_dir.mkdir(exist_ok=True)
self.ttl = settings.CACHE_TTL
def _get_cache_path(self, key: str) -> Path:
"""获取缓存文件路径"""
return self.cache_dir / f"{key}.json"
def get(self, key: str) -> Optional[Any]:
"""获取缓存数据"""
cache_path = self._get_cache_path(key)
if not cache_path.exists():
return None
try:
with open(cache_path, 'r', encoding='utf-8') as f:
data = json.load(f)
# 检查是否过期
cached_time = datetime.fromisoformat(data.get('_cached_at', '2000-01-01'))
if datetime.now() - cached_time > timedelta(seconds=self.ttl):
return None
return data.get('data')
except Exception:
return None
def set(self, key: str, data: Any) -> None:
"""设置缓存数据"""
cache_path = self._get_cache_path(key)
cache_data = {
'_cached_at': datetime.now().isoformat(),
'data': data
}
with open(cache_path, 'w', encoding='utf-8') as f:
json.dump(cache_data, f, ensure_ascii=False, indent=2, default=str)
def clear(self, key: Optional[str] = None) -> None:
"""清除缓存"""
if key:
cache_path = self._get_cache_path(key)
if cache_path.exists():
cache_path.unlink()
else:
# 清除所有缓存
for cache_file in self.cache_dir.glob('*.json'):
cache_file.unlink()
def clear_expired(self) -> int:
"""清除过期缓存,返回清除数量"""
count = 0
for cache_file in self.cache_dir.glob('*.json'):
try:
with open(cache_file, 'r', encoding='utf-8') as f:
data = json.load(f)
cached_time = datetime.fromisoformat(data.get('_cached_at', '2000-01-01'))
if datetime.now() - cached_time > timedelta(seconds=self.ttl):
cache_file.unlink()
count += 1
except Exception:
pass
return count
# 全局缓存实例
cache = CacheManager()

25
start.sh Executable file
View File

@@ -0,0 +1,25 @@
#!/bin/bash
# 启动福州江阴实船作业统计可视化系统
echo "🚀 启动 Confluence 数据采集与可视化系统..."
echo ""
# 激活虚拟环境
source venv/bin/activate
# 检查 .env 文件
if [ ! -f ".env" ]; then
echo "⚠️ .env 文件不存在,从 .env.example 复制..."
cp .env.example .env
echo "⚠️ 请编辑 .env 文件,填入你的 Confluence Token"
exit 1
fi
# 启动服务
echo "📊 启动 FastAPI 服务..."
echo "🔗 API 文档: http://localhost:8000/docs"
echo "🌐 可视化页面: http://localhost:8000"
echo ""
python -m uvicorn main:app --reload --host 0.0.0.0 --port 8000

3
static/README.md Normal file
View File

@@ -0,0 +1,3 @@
# Confluence Data Visualization System
See [README.md](../README.md) for full documentation.

334
static/index.html Normal file
View File

@@ -0,0 +1,334 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>福州江阴实船作业统计可视化</title>
<script src="https://cdn.jsdelivr.net/npm/echarts@5.4.3/dist/echarts.min.js"></script>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
background: #f5f7fa;
color: #333;
}
.header {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
color: white;
padding: 20px 40px;
box-shadow: 0 2px 10px rgba(0,0,0,0.1);
}
.header h1 {
font-size: 28px;
font-weight: 600;
}
.header .subtitle {
margin-top: 5px;
opacity: 0.9;
font-size: 14px;
}
.container {
max-width: 1400px;
margin: 0 auto;
padding: 30px 40px;
}
.kpi-cards {
display: grid;
grid-template-columns: repeat(4, 1fr);
gap: 20px;
margin-bottom: 30px;
}
.kpi-card {
background: white;
border-radius: 12px;
padding: 24px;
box-shadow: 0 2px 8px rgba(0,0,0,0.06);
transition: transform 0.2s, box-shadow 0.2s;
}
.kpi-card:hover {
transform: translateY(-2px);
box-shadow: 0 4px 12px rgba(0,0,0,0.1);
}
.kpi-card .label {
font-size: 13px;
color: #666;
margin-bottom: 8px;
}
.kpi-card .value {
font-size: 32px;
font-weight: 700;
color: #667eea;
}
.kpi-card .change {
font-size: 12px;
margin-top: 8px;
color: #52c41a;
}
.charts-grid {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 20px;
margin-bottom: 20px;
}
.chart-card {
background: white;
border-radius: 12px;
padding: 20px;
box-shadow: 0 2px 8px rgba(0,0,0,0.06);
}
.chart-card.full-width {
grid-column: span 2;
}
.chart-title {
font-size: 16px;
font-weight: 600;
margin-bottom: 15px;
color: #333;
}
.month-switcher {
display: inline-flex;
gap: 5px;
}
.month-btn {
background: #f0f0f0;
border: 1px solid #ddd;
padding: 4px 10px;
border-radius: 4px;
cursor: pointer;
font-size: 11px;
transition: all 0.2s;
}
.month-btn:hover {
background: #e0e0e0;
}
.month-btn.active {
background: #667eea;
color: white;
border-color: #667eea;
}
.eff-month-btn {
background: #f0f0f0;
border: 1px solid #ddd;
padding: 4px 10px;
border-radius: 4px;
cursor: pointer;
font-size: 11px;
transition: all 0.2s;
}
.eff-month-btn:hover {
background: #e0e0e0;
}
.eff-month-btn.active {
background: #764ba2;
color: white;
border-color: #764ba2;
}
.chart-container {
width: 100%;
height: 350px;
}
.chart-container.large {
height: 400px;
}
.loading {
display: flex;
justify-content: center;
align-items: center;
height: 300px;
color: #999;
}
.error {
background: #fff2f0;
border: 1px solid #ffccc7;
color: #ff4d4f;
padding: 16px;
border-radius: 8px;
margin: 20px 0;
}
.refresh-btn {
position: fixed;
bottom: 30px;
right: 30px;
background: #667eea;
color: white;
border: none;
padding: 14px 28px;
border-radius: 30px;
font-size: 14px;
cursor: pointer;
box-shadow: 0 4px 12px rgba(102, 126, 234, 0.4);
transition: all 0.3s;
}
.refresh-btn:hover {
background: #5a67d8;
transform: scale(1.05);
}
.refresh-btn:disabled {
background: #ccc;
cursor: not-allowed;
transform: none;
}
@media (max-width: 1024px) {
.charts-grid {
grid-template-columns: 1fr;
}
.chart-card.full-width {
grid-column: span 1;
}
.kpi-cards {
grid-template-columns: repeat(2, 1fr);
}
}
@media (max-width: 640px) {
.kpi-cards {
grid-template-columns: 1fr;
}
}
/* 指标切换按钮样式 */
.metric-btn {
background: #f0f2f5;
border: 1px solid #d9d9d9;
padding: 4px 12px;
margin-left: 5px;
border-radius: 4px;
cursor: pointer;
font-size: 12px;
transition: all 0.3s;
}
.metric-btn:hover {
background: #e6f7ff;
border-color: #667eea;
color: #667eea;
}
.metric-btn.active {
background: #667eea;
border-color: #667eea;
color: white;
}
</style>
</head>
<body>
<div class="header">
<h1>福州江阴实船作业统计</h1>
<div class="subtitle">数据驱动的作业分析与可视化报表</div>
</div>
<div class="container">
<!-- KPI 卡片 -->
<div class="kpi-cards">
<div class="kpi-card">
<div class="label">统计月份数</div>
<div class="value" id="kpi-months">-</div>
</div>
<div class="kpi-card">
<div class="label">船舶报告总数</div>
<div class="value" id="kpi-reports">-</div>
</div>
<div class="kpi-card">
<div class="label">活跃船次数</div>
<div class="value" id="kpi-ships">-</div>
</div>
<div class="kpi-card">
<div class="label">月均作业数</div>
<div class="value" id="kpi-avg">-</div>
</div>
</div>
<!-- 图表区域 -->
<div class="charts-grid">
<div class="chart-card">
<div class="chart-title">月度实船作业趋势</div>
<div id="trend-chart" class="chart-container"></div>
</div>
<div class="chart-card">
<div class="chart-title">每周作业对比统计</div>
<div id="weekly-stats-chart" class="chart-container"></div>
</div>
<div class="chart-card full-width">
<div class="chart-title">作业日历热力图</div>
<div id="calendar-chart" class="chart-container large"></div>
</div>
<div class="chart-card">
<div class="chart-title">
月度TEU排行榜
<div class="month-switcher" style="float: right; font-size: 12px;">
<button class="month-btn active" onclick="switchRankMonth('current')" id="btn-current">本月</button>
<button class="month-btn" onclick="switchRankMonth('last')" id="btn-last">上月</button>
</div>
</div>
<div id="efficiency-chart" class="chart-container"></div>
</div>
<div class="chart-card">
<div class="chart-title">
船次作业效率排行
<div class="month-switcher" style="float: right; font-size: 12px;">
<button class="eff-month-btn active" onclick="switchEfficiencyMonth('current')" id="eff-btn-current">本月</button>
<button class="eff-month-btn" onclick="switchEfficiencyMonth('last')" id="eff-btn-last">上月</button>
</div>
</div>
<div id="efficiency-rank-chart" class="chart-container"></div>
</div>
<div class="chart-card full-width">
<div class="chart-title">
每日船作业量对比
<span style="float: right; font-weight: normal; font-size: 12px;">
<button class="metric-btn" data-metric="ships" onclick="switchDailyMetric('ships')">船次</button>
<button class="metric-btn active" data-metric="teu" onclick="switchDailyMetric('teu')">TEU</button>
<button class="metric-btn" data-metric="moves" onclick="switchDailyMetric('moves')">Moves</button>
</span>
</div>
<div id="daily-comparison-chart" class="chart-container large"></div>
</div>
</div>
</div>
<button class="refresh-btn" id="refresh-btn" onclick="refreshData()">
刷新数据
</button>
<script src="js/charts.js?v=2"></script>
</body>
</html>

876
static/js/charts.js Normal file
View File

@@ -0,0 +1,876 @@
// 全局变量
let summaryData = null;
let allTableData = [];
let currentRankMonth = 'current';
let currentEffMonth = 'current';
let charts = {};
// API 基础 URL
const API_BASE = '/api';
// 统一配色方案
const COLORS = {
primary: '#667eea', // 主色:紫蓝
secondary: '#764ba2', // 辅色:紫色
light: '#e0e7ff', // 浅色:浅紫蓝
dark: '#4c51bf', // 深色:深紫蓝
gradient: ['#e0e7ff', '#c7d2fe', '#a5b4fc', '#818cf8', '#6366f1', '#4f46e5'],
background: '#f8fafc', // 背景色:灰白
text: '#334155', // 文字色:深灰
border: '#e2e8f0' // 边框色:浅灰
};
// 初始化
document.addEventListener('DOMContentLoaded', function() {
console.log('✅ Page loaded, starting data load...');
loadData();
});
// 加载数据
async function loadData() {
showLoading();
try {
console.log('📊 Loading summary data...');
const response = await fetch(`${API_BASE}/statistics/summary`);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
summaryData = await response.json();
console.log('✅ Summary loaded:', summaryData.total_ship_reports, 'ships');
updateKPIs(summaryData);
renderCharts(summaryData);
// 加载TEU排行数据
console.log('📊 Loading TEU ranking...');
await loadTEURankingData();
// 加载每日船作业量对比数据
console.log('📊 Loading daily stats...');
await loadDailyStats();
} catch (error) {
console.error('❌ 加载数据失败:', error);
showError('加载数据失败,请检查 API 服务是否正常运行');
}
}
// 刷新数据
async function refreshData() {
const btn = document.getElementById('refresh-btn');
btn.disabled = true;
btn.textContent = '刷新中...';
try {
const response = await fetch(`${API_BASE}/refresh`, { method: 'POST' });
if (response.ok) {
await loadData();
}
} catch (error) {
console.error('刷新失败:', error);
} finally {
btn.disabled = false;
btn.textContent = '刷新数据';
}
}
// 显示加载状态
function showLoading() {
const containers = document.querySelectorAll('.chart-container');
containers.forEach(container => {
container.innerHTML = '<div class="loading">数据加载中...</div>';
});
}
// 显示错误
function showError(message) {
const containers = document.querySelectorAll('.chart-container');
containers.forEach(container => {
container.innerHTML = `<div class="error">${message}</div>`;
});
}
// 更新 KPI 卡片
function updateKPIs(data) {
document.getElementById('kpi-months').textContent = data.total_months || 0;
document.getElementById('kpi-reports').textContent = data.total_ship_reports || 0;
document.getElementById('kpi-ships').textContent = Object.keys(data.ship_frequency || {}).length;
const avg = data.total_months > 0
? Math.round(data.total_ship_reports / data.total_months)
: 0;
document.getElementById('kpi-avg').textContent = avg;
}
// 渲染所有图表
function renderCharts(data) {
console.log('🎨 Rendering charts...');
renderTrendChart(data);
renderRecentChart(data);
// 注意:日历和每周统计在 loadDailyStats() 完成后调用
}
// 辅助函数十六进制颜色转RGBA
function hexToRgba(hex, alpha) {
const r = parseInt(hex.slice(1, 3), 16);
const g = parseInt(hex.slice(3, 5), 16);
const b = parseInt(hex.slice(5, 7), 16);
return `rgba(${r}, ${g}, ${b}, ${alpha})`;
}
// 辅助函数:获取系统当前月份(用于"本月"
function getCurrentMonth() {
const now = new Date();
const year = now.getFullYear();
const month = String(now.getMonth() + 1).padStart(2, '0');
return `${year}.${month}`;
}
// 辅助函数:根据当前月份计算上一个月
function getPreviousMonth(currentMonth) {
if (!currentMonth) return null;
const [year, month] = currentMonth.split('.').map(Number);
if (month === 1) {
return `${year - 1}.12`;
} else {
return `${year}.${String(month - 1).padStart(2, '0')}`;
}
}
// 1. 月度趋势图
function renderTrendChart(data) {
const chartDom = document.getElementById('trend-chart');
const chart = echarts.init(chartDom);
charts.trend = chart;
const months = data.monthly_trend.map(item => item.month);
const values = data.monthly_trend.map(item => item.total_ships);
const option = {
tooltip: {
trigger: 'axis',
axisPointer: {
type: 'cross'
}
},
grid: {
left: '3%',
right: '4%',
bottom: '3%',
containLabel: true
},
xAxis: {
type: 'category',
boundaryGap: false,
data: months,
axisLabel: {
rotate: 30
}
},
yAxis: {
type: 'value',
name: '作业次数',
minInterval: 1
},
series: [{
name: '实船作业数',
type: 'line',
smooth: true,
symbol: 'circle',
symbolSize: 8,
sampling: 'average',
itemStyle: {
color: COLORS.primary
},
areaStyle: {
color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [
{ offset: 0, color: hexToRgba(COLORS.primary, 0.3) },
{ offset: 1, color: hexToRgba(COLORS.primary, 0.05) }
])
},
data: values
}]
};
chart.setOption(option);
chart.on('click', function(params) {
const month = params.name;
window.open(`/api/monthly/${month}`, '_blank');
});
}
// 2. 作业日历热力图使用dailyStatsData
let dailyStatsData = [];
function renderCalendarChart() {
const chartDom = document.getElementById('calendar-chart');
if (!chartDom) {
console.log('⚠️ Calendar chart DOM not found');
return;
}
const chart = echarts.init(chartDom);
charts.calendar = chart;
if (!dailyStatsData || dailyStatsData.length === 0) {
console.log('⚠️ No daily stats data available');
chartDom.innerHTML = '<div class="loading">等待数据...</div>';
return;
}
console.log('📅 Rendering calendar with', dailyStatsData.length, 'days');
// 从dailyStatsData生成日历数据
const calendarData = dailyStatsData.map(day => [day.date, day.ship_count || 0]);
// 确定日期范围
const dates = dailyStatsData.map(d => d.date).sort();
const startStr = dates[0];
const endStr = dates[dates.length - 1];
// 计算最大值用于颜色映射
const maxCount = Math.max(...dailyStatsData.map(d => d.ship_count || 0), 1);
const option = {
tooltip: {
position: 'top',
formatter: function(params) {
const date = params.value[0];
const count = params.value[1];
const dayData = dailyStatsData.find(d => d.date === date);
if (count === 0) {
return `${date}<br/>无作业`;
}
return `${date}<br/>船次: ${count}<br/>TEU: ${dayData?.total_teu || 0}<br/>Moves: ${dayData?.total_moves || 0}`;
}
},
visualMap: {
min: 0,
max: maxCount,
calculable: true,
orient: 'horizontal',
left: 'center',
top: 'top',
inRange: {
color: COLORS.gradient
},
text: ['高', '无'],
textStyle: {
color: '#666'
}
},
calendar: {
top: 60,
left: 60,
right: 60,
cellSize: ['auto', 20],
range: [startStr, endStr],
itemStyle: {
borderWidth: 0.5,
borderColor: '#ddd'
},
yearLabel: { show: false },
dayLabel: {
firstDay: 1,
nameMap: 'cn'
},
monthLabel: {
nameMap: 'cn',
fontSize: 14,
fontWeight: 'bold'
},
splitLine: {
show: true,
lineStyle: {
color: '#eee'
}
}
},
series: [{
type: 'heatmap',
coordinateSystem: 'calendar',
data: calendarData,
itemStyle: {
borderWidth: 0.5,
borderColor: '#fff'
},
emphasis: {
itemStyle: {
shadowBlur: 10,
shadowColor: 'rgba(0, 0, 0, 0.5)'
}
}
}]
};
chart.setOption(option);
}
// 3. 每周作业对比统计
function renderWeeklyStatsChart() {
const chartDom = document.getElementById('weekly-stats-chart');
if (!chartDom) {
console.log('⚠️ Weekly chart DOM not found');
return;
}
const chart = echarts.init(chartDom);
charts.weekly = chart;
if (!dailyStatsData || dailyStatsData.length === 0) {
console.log('⚠️ No daily stats data for weekly chart');
chartDom.innerHTML = '<div class="loading">等待数据...</div>';
return;
}
// 按周汇总数据
const weeklyData = {};
dailyStatsData.forEach(day => {
const date = new Date(day.date);
const year = date.getFullYear();
// 获取ISO周数
const d = new Date(Date.UTC(year, date.getMonth(), date.getDate()));
const dayNum = d.getUTCDay() || 7;
d.setUTCDate(d.getUTCDate() + 4 - dayNum);
const yearStart = new Date(Date.UTC(d.getUTCFullYear(), 0, 1));
const weekNo = Math.ceil((((d - yearStart) / 86400000) + 1) / 7);
const weekKey = `${year}-W${weekNo.toString().padStart(2, '0')}`;
if (!weeklyData[weekKey]) {
weeklyData[weekKey] = {
week: weekKey,
ship_count: 0,
total_teu: 0,
total_moves: 0,
days: 0
};
}
weeklyData[weekKey].ship_count += day.ship_count || 0;
weeklyData[weekKey].total_teu += day.total_teu || 0;
weeklyData[weekKey].total_moves += day.total_moves || 0;
weeklyData[weekKey].days += 1;
});
// 转换为数组并排序
const sortedWeeklyData = Object.values(weeklyData).sort((a, b) => a.week.localeCompare(b.week));
const weeks = sortedWeeklyData.map(w => w.week);
const shipCounts = sortedWeeklyData.map(w => w.ship_count);
const teuValues = sortedWeeklyData.map(w => w.total_teu);
const movesValues = sortedWeeklyData.map(w => w.total_moves);
const option = {
tooltip: {
trigger: 'axis',
axisPointer: {
type: 'cross'
},
formatter: function(params) {
let result = params[0].name + '<br/>';
params.forEach(param => {
result += `${param.marker} ${param.seriesName}: ${param.value}<br/>`;
});
return result;
}
},
legend: {
data: ['船次', 'TEU', 'Moves'],
top: 0,
textStyle: {
fontSize: 11
}
},
grid: {
left: '3%',
right: '4%',
bottom: '12%',
top: '15%',
containLabel: true
},
dataZoom: [{
type: 'slider',
show: true,
xAxisIndex: [0],
start: Math.max(0, 100 - (12 / weeks.length * 100)),
end: 100,
height: 20,
bottom: 0
}, {
type: 'inside',
xAxisIndex: [0],
start: Math.max(0, 100 - (12 / weeks.length * 100)),
end: 100
}],
xAxis: {
type: 'category',
data: weeks,
axisLabel: {
rotate: 45,
fontSize: 10
}
},
yAxis: [
{
type: 'value',
name: '船次',
position: 'left',
axisLine: {
lineStyle: {
color: COLORS.primary
}
},
axisLabel: {
color: COLORS.primary
}
},
{
type: 'value',
name: 'TEU/Moves',
position: 'right',
axisLine: {
lineStyle: {
color: COLORS.secondary
}
},
axisLabel: {
color: COLORS.secondary
}
}
],
series: [
{
name: '船次',
type: 'bar',
data: shipCounts,
itemStyle: {
color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [
{ offset: 0, color: COLORS.primary },
{ offset: 1, color: hexToRgba(COLORS.primary, 0.6) }
])
},
yAxisIndex: 0
},
{
name: 'TEU',
type: 'line',
smooth: true,
data: teuValues,
itemStyle: {
color: COLORS.secondary
},
lineStyle: {
width: 2
},
yAxisIndex: 1
},
{
name: 'Moves',
type: 'line',
smooth: true,
data: movesValues,
itemStyle: {
color: COLORS.dark
},
lineStyle: {
width: 2,
type: 'dashed'
},
yAxisIndex: 1
}
]
};
chart.setOption(option);
}
// 4. 月度TEU排行榜横向柱状图
// 使用全局变量 currentRankMonth 和 allTableData
function switchRankMonth(month) {
currentRankMonth = month;
document.querySelectorAll('.month-btn').forEach(btn => {
btn.classList.remove('active');
});
document.getElementById(`btn-${month}`).classList.add('active');
renderEfficiencyChart();
}
async function loadTEURankingData() {
try {
const response = await fetch(`${API_BASE}/analysis/table-data`);
if (!response.ok) throw new Error('获取数据失败');
const data = await response.json();
allTableData = data.data || [];
console.log('✅ TEU ranking loaded:', allTableData.length, 'records');
renderEfficiencyChart();
renderEfficiencyRankChart();
} catch (error) {
console.error('❌ 加载TEU排行数据失败:', error);
}
}
function renderEfficiencyChart() {
const chartDom = document.getElementById('efficiency-chart');
const chart = echarts.init(chartDom);
charts.efficiency = chart;
// 动态获取最新月份
const latestMonth = getCurrentMonth();
if (!latestMonth) {
chartDom.innerHTML = '<div class="loading">暂无数据</div>';
return;
}
let targetMonth;
if (currentRankMonth === 'current') {
targetMonth = latestMonth;
} else {
targetMonth = getPreviousMonth(latestMonth);
}
const monthShips = allTableData.filter(ship => {
return ship.month === targetMonth && ship.teu !== null && ship.teu > 0;
});
const sortedShips = monthShips.sort((a, b) => b.teu - a.teu);
const top10 = sortedShips.slice(0, 10);
const shipLabels = top10.map(ship => {
const shipNum = ship.ship_code.replace('FZ ', '');
const shipName = ship.ship_name || '未知';
return `${shipNum} ${shipName}`;
});
const teuValues = top10.map(ship => ship.teu);
const option = {
tooltip: {
trigger: 'axis',
axisPointer: {
type: 'shadow'
},
formatter: function(params) {
const ship = top10[params[0].dataIndex];
return `${params[0].name}<br/>TEU: ${ship.teu}<br/>Moves: ${ship.moves || 'N/A'}`;
}
},
grid: {
left: '3%',
right: '8%',
bottom: '3%',
containLabel: true
},
xAxis: {
type: 'value',
name: 'TEU',
axisLabel: {
formatter: '{value}'
}
},
yAxis: {
type: 'category',
data: shipLabels,
axisLabel: {
fontSize: 12,
width: 120,
overflow: 'truncate'
},
inverse: true
},
series: [{
name: 'TEU',
type: 'bar',
data: teuValues,
itemStyle: {
color: new echarts.graphic.LinearGradient(0, 0, 1, 0, [
{ offset: 0, color: COLORS.primary },
{ offset: 1, color: COLORS.secondary }
])
},
label: {
show: true,
position: 'right',
formatter: '{c}'
}
}]
};
chart.setOption(option, true);
}
// 5. 船次作业效率排行(净效率)
function switchEfficiencyMonth(month) {
currentEffMonth = month;
document.querySelectorAll('.eff-month-btn').forEach(btn => {
btn.classList.remove('active');
});
document.getElementById(`eff-btn-${month}`).classList.add('active');
renderEfficiencyRankChart();
}
function renderEfficiencyRankChart() {
const chartDom = document.getElementById('efficiency-rank-chart');
if (!chartDom) {
console.log('⚠️ Efficiency rank chart DOM not found');
return;
}
const chart = echarts.init(chartDom);
charts.efficiencyRank = chart;
// 动态获取最新月份
const latestMonth = getCurrentMonth();
if (!latestMonth) {
chartDom.innerHTML = '<div class="loading">暂无数据</div>';
return;
}
let targetMonth;
if (currentEffMonth === 'current') {
targetMonth = latestMonth;
} else {
targetMonth = getPreviousMonth(latestMonth);
}
// 筛选有净效率数据的船舶
const monthShips = allTableData.filter(ship => {
return ship.month === targetMonth &&
ship.net_efficiency !== null &&
ship.net_efficiency > 0;
});
console.log(`📊 Efficiency rank for ${targetMonth}: ${monthShips.length} ships with data`);
if (monthShips.length === 0) {
chartDom.innerHTML = '<div class="loading">该月份无效率数据</div>';
return;
}
// 按净效率降序排列取Top 10
const sortedShips = monthShips.sort((a, b) => b.net_efficiency - a.net_efficiency);
const top10 = sortedShips.slice(0, 10);
const shipLabels = top10.map(ship => {
const shipNum = ship.ship_code.replace('FZ ', '');
const shipName = ship.ship_name || '未知';
return `${shipNum} ${shipName}`;
});
const efficiencyValues = top10.map(ship => ship.net_efficiency);
const option = {
tooltip: {
trigger: 'axis',
axisPointer: {
type: 'shadow'
},
formatter: function(params) {
const ship = top10[params[0].dataIndex];
return `${params[0].name}<br/>净效率: ${ship.net_efficiency} move/车/小时<br/>毛效率: ${ship.gross_efficiency || 'N/A'}`;
}
},
grid: {
left: '3%',
right: '8%',
bottom: '3%',
containLabel: true
},
xAxis: {
type: 'value',
name: '净效率(move/车/小时)',
axisLabel: {
formatter: '{value}'
}
},
yAxis: {
type: 'category',
data: shipLabels,
axisLabel: {
fontSize: 12,
width: 120,
overflow: 'truncate'
},
inverse: true
},
series: [{
name: '净效率',
type: 'bar',
data: efficiencyValues,
itemStyle: {
color: new echarts.graphic.LinearGradient(0, 0, 1, 0, [
{ offset: 0, color: COLORS.secondary },
{ offset: 1, color: COLORS.dark }
])
},
label: {
show: true,
position: 'right',
formatter: '{c}'
}
}]
};
chart.setOption(option, true);
}
// 6. 最近作业记录(时间线)- 保留函数但不再使用
function renderRecentChart(data) {
// 已移除,保留空函数避免错误
console.log('Recent chart removed');
}
// 每日船作业量对比图表
let currentDailyMetric = 'teu';
function switchDailyMetric(metric) {
currentDailyMetric = metric;
// 更新按钮状态
document.querySelectorAll('.metric-btn').forEach(btn => {
btn.classList.remove('active');
});
document.querySelector(`[data-metric="${metric}"]`).classList.add('active');
renderDailyComparisonChart();
}
async function loadDailyStats() {
console.log('📊 Loading daily stats...');
try {
const response = await fetch(`${API_BASE}/analysis/daily-stats`);
if (!response.ok) throw new Error('获取数据失败');
const data = await response.json();
dailyStatsData = data.daily_stats || [];
console.log('✅ Daily stats loaded:', dailyStatsData.length, 'days');
// 重新渲染日历和每周统计
renderCalendarChart();
renderWeeklyStatsChart();
renderDailyComparisonChart();
} catch (error) {
console.error('❌ 加载每日统计数据失败:', error);
}
}
function renderDailyComparisonChart() {
const chartDom = document.getElementById('daily-comparison-chart');
if (!chartDom) return;
const chart = echarts.init(chartDom);
charts.daily = chart;
if (dailyStatsData.length === 0) {
chartDom.innerHTML = '<div class="loading">等待数据...</div>';
return;
}
// 准备数据
const dates = dailyStatsData.map(d => d.date);
let values, yAxisName, seriesName;
switch(currentDailyMetric) {
case 'teu':
values = dailyStatsData.map(d => d.total_teu || 0);
yAxisName = 'TEU';
seriesName = 'TEU';
break;
case 'moves':
values = dailyStatsData.map(d => d.total_moves || 0);
yAxisName = 'Moves';
seriesName = 'Moves';
break;
default: // ships
values = dailyStatsData.map(d => d.ship_count || 0);
yAxisName = '船次';
seriesName = '船次';
}
const option = {
tooltip: {
trigger: 'axis',
axisPointer: {
type: 'cross'
},
formatter: function(params) {
const day = dailyStatsData[params[0].dataIndex];
return `${day.date}<br/>船次: ${day.ship_count}<br/>TEU: ${day.total_teu || 0}<br/>Moves: ${day.total_moves || 0}`;
}
},
grid: {
left: '3%',
right: '4%',
bottom: '15%',
containLabel: true
},
dataZoom: [{
type: 'slider',
show: true,
xAxisIndex: [0],
start: 70,
end: 100,
height: 20,
bottom: 10
}, {
type: 'inside',
xAxisIndex: [0],
start: 70,
end: 100
}],
xAxis: {
type: 'category',
data: dates,
axisLabel: {
rotate: 45,
fontSize: 10
}
},
yAxis: {
type: 'value',
name: yAxisName,
minInterval: 1
},
series: [{
name: seriesName,
type: 'line',
smooth: true,
symbol: 'circle',
symbolSize: 6,
data: values,
itemStyle: {
color: COLORS.primary
},
lineStyle: {
width: 2,
color: COLORS.primary
},
areaStyle: {
color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [
{ offset: 0, color: hexToRgba(COLORS.primary, 0.3) },
{ offset: 1, color: hexToRgba(COLORS.primary, 0.05) }
])
}
}]
};
chart.setOption(option);
}
// 窗口大小改变时重绘图表
window.addEventListener('resize', function() {
Object.values(charts).forEach(chart => {
if (chart && chart.resize) {
chart.resize();
}
});
});