如何在Kubernetes (AKS)中使用Elastic APM、Metricbeat和Filebeat安装ELK？

Question

我可以安装elasticseach和kibana，它们都已启动并运行。在Kibana仪表板上设置了APM服务器，并显示了索引。

当我跟踪日志时，我得到了APM-Agent的以下错误。错误co.elastic.apm.agent.report.IntakeV2ReportingEventHandler -尝试连接到APM服务器时出错。与当前连接对应的SSL配置的一些详细信息被记录在INFO级别。

错误co.elastic.apm.agent.report.IntakeV2ReportingEventHandler -无法处理JSON_WRITER类型的事件，出现此错误:连接超时

APM-代理Yaml文件

apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
namespace: blogdemodeployments
spec:  
selector:
matchLabels:
app: azuretest
template:
metadata:
labels:
app: azuretest
spec:
containers:
      - name: apm-agent-container
image: dockerid/application-service
volumeMounts:
        - name: shared-data
mountPath: /usr/share/app/
ports:
        - containerPort: 6000
name: http
protocol: TCP
env:
        - name: SERVER_URL
value: "http://40.83.185.238:8200"
      - name: filebeat-container
image: docker.elastic.co/beats/filebeat:7.10.0
volumeMounts:
        - name: shared-data
mountPath: /usr/share/filebeat/filebeat.yml

volumes:
      - name: shared-data
azureFile:
secretName: storage-secret
shareName: myfileshare
readOnly: false
 
---
kind: Service
apiVersion: apps/v1
metadata:
name: apmfb
namespace: blogdemodeployments
spec:
type: LoadBalancer
selector:
app: apmfb
ports:
    - name: http
protocol: TCP
port: 6000
targetPort: 6000

ElasticSearch Yaml

---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
  labels:
    component: elasticsearch
spec:
  version: 7.10.2
  http:
    service:
      spec:
        type: LoadBalancer
  nodeSets:
  - name: default
    count: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false

Kibana Yaml

---
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: quickstart
spec:
  version: 7.10.2 #Make sure Kibana and Elasticsearch are on the same version.
  http:
    service:
      spec:
        type: LoadBalancer #Adds a External IP
    tls:
      selfSignedCertificate:
        disabled: true
  count: 1
  elasticsearchRef:
    name: quickstart

APM服务器Yaml

---
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
  name: apm-new-quickstart
  namespace: default
spec:
  version: 7.10.2
  count: 1
  http:
    service:
      spec:
        type: LoadBalancer
    tls:
      selfSignedCertificate:
        disabled: true
  config:
    output:
      elasticsearch:
        enabled: true
        hosts: ["https://52.224.33.53:9200"]
        username: "elastic"
        password: "XXXXXXXXXXXXXXXXXXXX"
    logging:
      level: debug
      to_files: false
      to_stderr: true
    apm-server:
      hosts: "0.0.0.0:8200"

Answer 1

该错误表明代理无法连接到apm-server。在apm-agent-container环境中，SERVER_URL应该是ELASTIC_APM_SERVER_URL。

Answer 2

感谢您的回复，我可以将apm-server与代理连接起来，但在kibana仪表板中，我得到的信息是“尚未收到来自代理的数据”。我的应用程序运行正常

2021-03-25 08:47:06,605 [main] INFO co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean
2021-03-25 08:47:06,676 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.19.0 as demo-service1 on Java 10.0.2 Runtime version: 10.0.2+13-Debian-2 VM version: 10.0.2+13-Debian-2 (Oracle Corporation) Linux 5.4.0-1040-azure
2021-03-25 08:47:06,676 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - VM Arguments: [-javaagent:/app/elastic-apm-agent-1.19.0.jar, -Delastic.apm.service_name=demo-service1, -Delastic.apm.application_packages=com.javadeveloperzone.log4j, -Delastic.apm.server_urls=, -Delastic.apm.enable_log_correlation=true]
2021-03-25 08:47:07,836 [main] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state

 . ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.0.3.RELEASE)
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.cglib.core.ReflectUtils$1 (jar:file:/app/app.jar!/BOOT-INF/lib/spring-core-5.0.7.RELEASE.jar!/) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of org.springframework.cglib.core.ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2021-03-25 08:47:12,189 [main] INFO co.elastic.apm.agent.servlet.ServletVersionInstrumentation - Servlet container info = Apache Tomcat/8.5.31
2021-03-25 08:52:15,083 INFO [Log4JExample] c979238deffe75982fe43844c268c83e - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 08:52:15,083 INFO [Log4JExample] c979238deffe75982fe43844c268c83e - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 08:52:20,706 INFO [Log4JExample] 3cb508519b2d44f9c5cab6da497d0745 - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 08:52:20,706 INFO [Log4JExample] 3cb508519b2d44f9c5cab6da497d0745 - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]
2021-03-25 10:46:03,993 INFO [Log4JExample] 2828f071583f306e89591a4a2edf2080 - Demobject [productid=qert, productname=qwerty, price=50.0, description=qwerty sdfghj zxcvb]