Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ClsToES demo #161

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions Python3.6-ClsToElasticSearch/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"serverless-cloud-function-application": {
"Chinese": {
"name": "CLS 消息转储至 ES",
"description": "使用CLS+云函数+ES+Kibana,提供日志转存能力,替代Logstash,成本更低,支持功能自定义。代码里面可以添加日志清洗、转换等功能",
"attention": "在使用本示例代码时需要在函数配置里添加环境变量和VPC配置;环境变量:ES_ADDRESS,ES_USER,ES_PASSWORD,ES_API_KEY,ES_INDEX。本示例代码仅供参考,如需用于生产请根据自身业务进行相应的评估。",
"tutorial":"https://cloud.tencent.com/document/product/583/51597",
"author": {
"name": "腾讯云"
}
},
"English": {
"name": "CLSToElasticsearch",
"description": "This demo will connect CLS and consume message automatically.",
"attention": "When using this demo, you need to add environment variables and VPC configuration in the function configuration; environment variables: ES_ADDRESS, ES_USER, ES_PASSWORD, ES_API_KEY, ES_INDEX.",
"tutorial":" ",
"author": {
"name": "Tencent Cloud"
}
},
"runtime": "Python3.6",
"readme": "https://github.com/tencentyun/serverless-demo/tree/master/Python3.6-ClsToElasticSearch",
"version": "1.10.4",
"tags": [
"Python3.6",
"CLS",
"ES",
"Logstash",
"ETL"
]
}
}
30 changes: 30 additions & 0 deletions Python3.6-ClsToElasticSearch/serverless.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
component: scf
name: ap-guangzhou_default_CLSToElasticsearch
inputs:
name: CLSToElasticsearch
src: ./src
handler: index.main_handler
runtime: Python3.6
namespace: default
region: ap-guangzhou
memorySize: 256
environment:
- key: ES_ADDRESS
value: ES address/ES 地址
- key: ES_USER
value: ES user name/ ES 用户名
- key: ES_PASSWORD
value: ES password/ES 密码
- key: ES_INDEX
value: ES index
- key: ES_API_KEY
value: ES API KEY
timeout: 30
triggers:
- type: cls
name:
enable: true
topic:
maxMsgNum: 50
offset: latest
retry: 10000
62 changes: 62 additions & 0 deletions Python3.6-ClsToElasticSearch/src/elasticsearch/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# flake8: noqa
from __future__ import absolute_import

VERSION = (7, 7, 0)
__version__ = VERSION
__versionstr__ = "7.7.0"

import logging
import warnings

logger = logging.getLogger("elasticsearch")
logger.addHandler(logging.NullHandler())

from .client import Elasticsearch
from .transport import Transport
from .connection_pool import ConnectionPool, ConnectionSelector, RoundRobinSelector
from .serializer import JSONSerializer
from .connection import Connection, RequestsHttpConnection, Urllib3HttpConnection
from .exceptions import (
ImproperlyConfigured,
ElasticsearchException,
SerializationError,
TransportError,
NotFoundError,
ConflictError,
RequestError,
ConnectionError,
SSLError,
ConnectionTimeout,
AuthenticationException,
AuthorizationException,
ElasticsearchDeprecationWarning,
)

# Only raise one warning per deprecation message so as not
# to spam up the user if the same action is done multiple times.
warnings.simplefilter("default", category=ElasticsearchDeprecationWarning, append=True)

__all__ = [
"Elasticsearch",
"Transport",
"ConnectionPool",
"ConnectionSelector",
"RoundRobinSelector",
"JSONSerializer",
"Connection",
"RequestsHttpConnection",
"Urllib3HttpConnection",
"ImproperlyConfigured",
"ElasticsearchException",
"SerializationError",
"TransportError",
"NotFoundError",
"ConflictError",
"RequestError",
"ConnectionError",
"SSLError",
"ConnectionTimeout",
"AuthenticationException",
"AuthorizationException",
"ElasticsearchDeprecationWarning",
]
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading