본 포스팅은 패스트캠퍼스 환급 챌린지 참여를 위해 작성하였습니다.
https://bit.ly/4hTSJNB
I. 학습 인증샷 4장 이상 포함
1. 공부시작: 날짜, 시각 다 나오도록
2. 공부 종료: 날짜 시각 다 나오도록
3. 1개 클립 수강 (강의장 목록 캡쳐, 수강화면이 보이지 않도록) 1장
4. 학습 인증샷 1장(필기 촬영이나 작업물 촬영)
II. 학습 후기 700자 이상 (공백 제외)
실습이다~!
요약
저장소 clone
- 저장소를 fork한후 clone
# repo clone
Cloning into 'fastcampus-kafka-message-queue'...
remote: Enumerating objects: 533, done.
remote: Counting objects: 100% (58/58), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 533 (delta 46), reused 43 (delta 43), pack-reused 475 (from 1)
Receiving objects: 100% (533/533), 1.16 MiB | 1.15 MiB/s, done.
Resolving deltas: 100% (172/172), done.
ahn@Ahns-m1Pro14 rkaehdaos %Docker Compose 이미지 다운로드
- 이미지 미리 다운로드 - pull
# 도커 이미지 다운로드
ahn@Ahns-m1Pro14 fastcampus-kafka-message-queue % docker compose pull
WARN[0000] /Users/ahn/dev/repo/rkaehdaos/fastcampus-kafka-message-queue/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Pulling 5/42
✔ kafka1 Skipped - Image is already being pulled by kafka3 0.0s
✔ kafka2 Skipped - Image is already being pulled by kafka3 0.0s
⠧ zookeeper [⠀] Pulling 21.8s
⠧ cmak [⣄⣶⣿⣿⣿⠀⠀] Pulling 21.8s
⠧ redpanda-console [⠀⠀⠀⠀⠀⠀⠀] Pulling 21.8s
⠧ kafka3 [⠀] Pulling 21.8s
⠧ kafka-ui [⠀⠀⠀⠀⠀⠀⠀] Pulling 21.8s
⠧ mysql [⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀] Pulling 21.8s2 docker 관련 커맨드
- docker compose up -d
- compose 환경 올리기
- -d 옵션은 background 옵션
- 필요할때만 로그를 보도록 하자
ahn@Ahns-m1Pro14 fastcampus-kafka-message-queue % docker compose up -d
WARN[0000] /Users/ahn/dev/repo/rkaehdaos/fastcampus-kafka-message-queue/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Running 8/8
✔ Container mysql Running 0.0s
✔ Container zookeeper Running 0.0s
✔ Container kafka1 Running 0.0s
✔ Container kafka3 Running 0.0s
✔ Container kafka2 Running 0.0s
✔ Container cmak Running 0.0s
✔ Container kafka-ui Running 0.0s
✔ Container redpanda-console Running 0.0s
ahn@Ahns-m1Pro14 fastcampus-kafka-message-queue %- docker compose down
- 환경 내리기
- 실습 끝나면 실행
ahn@Ahns-m1Pro14 fastcampus-kafka-message-queue % docker compose down
WARN[0000] /Users/ahn/dev/repo/rkaehdaos/fastcampus-kafka-message-queue/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Running 9/9
✔ Container mysql Removed 1.3s
✔ Container cmak Removed 0.6s
✔ Container redpanda-console Removed 0.2s
✔ Container kafka-ui Removed 2.3s
✔ Container kafka1 Removed 1.0s
✔ Container kafka2 Removed 5.9s
✔ Container kafka3 Removed 1.0s
✔ Container zookeeper Removed 0.6s
✔ Network fastcampus-kafka-message-queue_default Removed 0.3s
ahn@Ahns-m1Pro14 fastcampus-kafka-message-queue %- docker ps
- 컨테이너 리스트
- container id 를 확인가능
ahn@Mac study-kafka-message-queue % docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eddf825abbd0 provectuslabs/kafka-ui:v0.7.1 "/bin/sh -c 'java --…" 25 seconds ago Up 23 seconds 0.0.0.0:8081->8080/tcp kafka-ui
2cd28ccba6aa docker.redpanda.com/redpandadata/console:v2.3.7 "./console" 25 seconds ago Up 23 seconds 0.0.0.0:8989->8080/tcp redpanda-console
0487f9aaa9b0 hlebalbau/kafka-manager:3.0.0.5 "/kafka-manager/bin/…" 25 seconds ago Up 23 seconds 0.0.0.0:9000->9000/tcp cmak
a62592fd749e bitnami/kafka:3.6.0 "/opt/bitnami/script…" 25 seconds ago Up 23 seconds 9092/tcp, 0.0.0.0:9094->9094/tcp, 0.0.0.0:54745->19092/tcp kafka3
9160af910160 bitnami/kafka:3.6.0 "/opt/bitnami/script…" 25 seconds ago Up 23 seconds 0.0.0.0:9092->9092/tcp, 0.0.0.0:54742->19092/tcp kafka1
005918c311ce bitnami/kafka:3.6.0 "/opt/bitnami/script…" 25 seconds ago Up 23 seconds 9092/tcp, 0.0.0.0:9093->9093/tcp, 0.0.0.0:54746->19092/tcp kafka2
db515dd6ee89 mysql:8.0.35 "docker-entrypoint.s…" 25 seconds ago Up 23 seconds 0.0.0.0:3306->3306/tcp, 33060/tcp mysql
619b243ab294 bitnami/zookeeper:3.7.2 "/opt/bitnami/script…" 25 seconds ago Up 23 seconds 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp zookeeper
ahn@Mac study-kafka-message-queue %- docker logs {컨테이너id}
- 특정 컨테이너 id를 지정해서 로그를 볼 수 있다.
# ex) docker ui 컨테이너의 로그 보기
ahn@Mac study-kafka-message-queue % docker logs eddf825abbd0
02:08:12,807 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This is logback-classic version 1.4.7
02:08:12,834 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
02:08:12,834 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
02:08:12,841 |-INFO in ch.qos.logback.classic.BasicConfigurator@e15b7e8 - Setting up default configuration.
02:08:13,700 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@1b2abca6 - URL [jar:file:/kafka-ui-api.jar!/BOOT-INF/classes!/logback-spring.xml] is not of type file
02:08:13,824 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [STDOUT]
02:08:13,824 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
02:08:13,859 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - This appender no longer admits a layout as a sub-component, set an encoder instead.
02:08:13,859 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder.
02:08:13,859 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - See also http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details
02:08:13,860 |-INFO in ch.qos.logback.classic.model.processor.RootLoggerModelHandler - Setting level of ROOT logger to INFO
02:08:13,860 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@6392827e - Propagating INFO level on Logger[ROOT] onto the JUL framework
02:08:13,860 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [STDOUT] to Logger[ROOT]
02:08:13,860 |-INFO in ch.qos.logback.core.model.processor.DefaultProcessor@2ed2d9cb - End of configuration.
02:08:13,860 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator@d5b810e - Registering current configuration as safe fallback point
_ _ ___ __ _ _ _ __ __ _
| | | |_ _| / _|___ _ _ /_\ _ __ __ _ __| |_ ___ | |/ /__ _ / _| |_____
| |_| || | | _/ _ | '_| / _ \| '_ / _` / _| ' \/ -_) | ' </ _` | _| / / _`|
\___/|___| |_| \___|_| /_/ \_| .__\__,_\__|_||_\___| |_|\_\__,_|_| |_\_\__,|
|_|
2025-04-07 02:08:13,919 INFO [background-preinit] o.h.v.i.u.Version: HV000001: Hibernate Validator 8.0.0.Final
2025-04-07 02:08:14,022 INFO [main] c.p.k.u.KafkaUiApplication: Starting KafkaUiApplication using Java 17.0.6 with PID 1 (/kafka-ui-api.jar started by kafkaui in /)
2025-04-07 02:08:14,022 DEBUG [main] c.p.k.u.KafkaUiApplication: Running with Spring Boot v3.0.6, Spring v6.0.8
2025-04-07 02:08:14,023 INFO [main] c.p.k.u.KafkaUiApplication: No active profile set, falling back to 1 default profile: "default"
2025-04-07 02:08:16,472 DEBUG [main] c.p.k.u.s.SerdesInitializer: Configuring serdes for cluster local
2025-04-07 02:08:16,949 INFO [main] o.s.b.a.e.w.EndpointLinksResolver: Exposing 2 endpoint(s) beneath base path '/actuator'
2025-04-07 02:08:16,979 INFO [main] o.s.b.a.s.r.ReactiveUserDetailsServiceAutoConfiguration:
Using generated security password: ea329ee7-fe3f-4a3a-a2d1-6ca6a90572fb
2025-04-07 02:08:17,107 WARN [main] c.p.k.u.c.a.DisabledAuthSecurityConfig: Authentication is disabled. Access will be unrestricted.
2025-04-07 02:08:17,378 INFO [main] o.s.b.w.e.n.NettyWebServer: Netty started on port 8080
2025-04-07 02:08:17,401 INFO [main] c.p.k.u.KafkaUiApplication: Started KafkaUiApplication in 4.307 seconds (process running for 5.057)
2025-04-07 02:08:18,006 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: local
2025-04-07 02:08:18,012 INFO [parallel-2] o.a.k.c.a.AdminClientConfig: AdminClientConfig values:
bootstrap.servers = [kafka1:19092, kafka2:19092, kafka3:19092]
client.dns.lookup = use_all_dns_ips
client.id = kafka-ui-admin-1743991698-1
connections.max.idle.ms = 300000
default.api.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 2147483647
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.connect.timeout.ms = null
sasl.login.read.timeout.ms = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.login.retry.backoff.max.ms = 10000
sasl.login.retry.backoff.ms = 100
sasl.mechanism = GSSAPI
sasl.oauthbearer.clock.skew.seconds = 30
sasl.oauthbearer.expected.audience = null
sasl.oauthbearer.expected.issuer = null
sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000
sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000
sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100
sasl.oauthbearer.jwks.endpoint.url = null
sasl.oauthbearer.scope.claim.name = scope
sasl.oauthbearer.sub.claim.name = sub
sasl.oauthbearer.token.endpoint.url = null
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
socket.connection.setup.timeout.max.ms = 30000
socket.connection.setup.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ssl.endpoint.identification.algorithm = https
ssl.engine.factory.class = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.certificate.chain = null
ssl.keystore.key = null
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.3
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.certificates = null
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
2025-04-07 02:08:18,045 INFO [parallel-2] o.a.k.c.u.AppInfoParser: Kafka version: 3.3.1
2025-04-07 02:08:18,045 INFO [parallel-2] o.a.k.c.u.AppInfoParser: Kafka commitId: e23c59d00e687ff5
2025-04-07 02:08:18,045 INFO [parallel-2] o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1743991698044
2025-04-07 02:08:18,279 DEBUG [parallel-10] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: local
2025-04-07 02:08:47,419 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: local
2025-04-07 02:08:47,460 DEBUG [parallel-6] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: local
2025-04-07 02:09:17,405 DEBUG [parallel-7] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: local
2025-04-07 02:09:17,437 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: local
2025-04-07 02:09:47,407 DEBUG [parallel-3] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: local
2025-04-07 02:09:47,453 DEBUG [parallel-8] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: local
ahn@Mac study-kafka-message-queue %- docker compose exec {컨테이너 서비스명} {컨테이너 안에서 실행할 명령어}
- 특정 컨테이너 내부에서 명령어 실행 하기
- error 발생
ahn@Mac study-kafka-message-queue % docker compose exec kafka1 kafka-topics.sh --create --topic my-topic --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --replication-factor 1 --partitions 1
[2025-04-07 03:31:03,427] WARN [AdminClient clientId=adminclient-1] Connection to node 3 (localhost/127.0.0.1:9094) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2025-04-07 03:31:03,428] WARN [AdminClient clientId=adminclient-1] Connection to node 2 (localhost/127.0.0.1:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)해당 부분은 수정 후 패캠 커뮤니티에 올렸다. (나같은 삽질 하지말라고.. ㅋㅋ)
https://fastcampus.co.kr/community/100810
파트2 에서 topic 안만들어지시는 분들.(특히 맥) | 패스트캠퍼스
패스트캠퍼스 커뮤니티 | 질문/답변
fastcampus.co.kr
아 실습.. 기쁘다..
'패캠챌린지 > Kafka EcoSystem - 진행중' 카테고리의 다른 글
| 패스트캠퍼스 환급챌린지 35일차 : 한번에 끝내는 KafkaEcosystem 강의 후기 (0) | 2025.04.08 |
|---|---|
| 패스트캠퍼스 환급챌린지 34일차 : 한번에 끝내는 KafkaEcosystem 강의 후기 (0) | 2025.04.07 |
| 패스트캠퍼스 환급챌린지 32일차 : 한번에 끝내는 KafkaEcosystem 강의 후기 (0) | 2025.04.05 |
| 패스트캠퍼스 환급챌린지 31일차 : 한번에 끝내는 KafkaEcosystem 강의 후기 (0) | 2025.04.04 |
| 패스트캠퍼스 환급챌린지 30일차 : 한번에 끝내는 KafkaEcosystem 강의 후기 (0) | 2025.04.03 |
감동맨
rkaehdaos의 블로그