기존에 AWS EC2의 m3.large에 jira, confluence를 설치형으로 운영하고 있었다.

(nginx, mysql도 같은 인스턴스에 설치하여 운용)

 

최근들어서 메모리 부족 이슈가 자꾸 발생한다.

  •  java.lang.OutOfMemoryError: Java heap space

일단은 스케줄 작업 시간을 분산해보았다.

confluence schedule job

 

몇일 괜찮더니 여전히 또 메모리 부족으로 confluence 접근이 안된다.

EC2 사양을 높이고 m3.large -> m3.xlarge로 (7.5G -> 15G) 메모리를 높였다.

 

메모리 사용하는 프로세스

  • java/confluence
  • java/jira
  • nginx
  • mysql

 

startup.sh 명령을 실행해서 띄웠다.

 

안되고 500 ERROR가 난다.

29-Jul-2019 01:51:06.145 SEVERE [http-nio-8090-exec-2] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=500, location=/500page.jsp]

 com.atlassian.util.concurrent.LazyReference$InitializationException: java.lang.NullPointerException

        at com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:149)

        at com.atlassian.util.concurrent.LazyReference.get(LazyReference.java:112)

        at com.atlassian.confluence.plugin.servlet.filter.ServletFilterModuleContainerFilter.getServletModuleManager(ServletFilterModuleContainerFilter.java:23)

        at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:68)

        at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)

        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)

        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

        at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:50)

        at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)

        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)

        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

        at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:721)

        at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:468)

        at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:391)

        at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:318)

        at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:439)

        at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:305)

        at org.apache.catalina.core.StandardHostValve.throwable(StandardHostValve.java:399)

        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)

        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)

        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)

        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)

        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)

        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)

        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521)

        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

        at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.NullPointerException

        at com.atlassian.spring.container.ContainerManager.getComponent(ContainerManager.java:33)

        at com.atlassian.confluence.util.LazyComponentReference$Accessor.get(LazyComponentReference.java:46)

        at com.atlassian.util.concurrent.Lazy$Strong.create(Lazy.java:85)

        at com.atlassian.util.concurrent.LazyReference$Sync.run(LazyReference.java:321)

        at com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:143)

        ... 29 more

 

아...... 일단 구글링

https://www.google.com/search?newwindow=1&ei=pFg-XcOuMpXMvwTrhKbABA&q=org.apache.catalina.core.StandardHostValve.custom+Exception+Processing+ErrorPage%5BerrorCode%3D500%2C+location%3D%2F500page.jsp%5D&oq=org.apache.catalina.core.StandardHostValve.custom+Exception+Processing+ErrorPage%5BerrorCode%3D500%2C+location%3D%2F500page.jsp%5D&gs_l=psy-ab.3...12614.12614..12886...0.0..0.0.0.......0....2j1..gws-wiz.Xc4qbAH0AaA&ved=0ahUKEwjDkOSgidnjAhUV5o8KHWuCCUgQ4dUDCAo&uact=5

 

org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=500, location=/500page.jsp] - Google

2013. 5. 24. · Well, we get a 500 error on the home page after starting up at least. The error we see in the ... StandardHostValve custom SEVERE: Exception Processing ErrorPage[errorCode=500, location=/500page.jsp] java.lang. NullPointerException at ... do

www.google.com

 

다행히 1번째 결과가 나의 상황과 유사하다.

링크를 열어보니 희망이 생긴다.

https://community.atlassian.com/t5/Confluence-questions/500-Error-on-Confluence-Startup/qaq-p/393445

 

500 Error on Confluence Startup

I am unable to start up our Confluence instance. Well, we get a 500 error on the home page after starting up at least. The error we see in the web page is: exception com.atlassian.util.concurrent.LazyReference$InitializationException: java.lang.NullPointer

community.atlassian.com

 

링크의 댓글을 따라가본다. (That did it for me, thanks! 라고 한다. 이거 왠지 될거 같다.)

 

How to clear Confluence plugins cache - Atlassian Documentation

How to clear Confluence plugins cache

confluence.atlassian.com

왠지 느낌이 이거같다. 플러그인 캐쉬 문제

 

solution for this error

다 풀릴것 같은 문제가 <confluence-home>을 못찾고 또 지연된다.

결국 find로 뒤져보니 저 디렉토리가 똑같이 존재한다.

 

언급된 디렉토리를 삭제하진 않고 일단 mv 명령어로 다른 path로 옮기고

confluence를 재시작하였다.

 

시작하면서 저 디렉토리들을 새로 만든다.

그러다보니 기동하는데 오래걸린다.  (참고로 default 8090 port로 띄움)

 

띄우고나서 확인, 아까는 500 ERROR였는데 200 OK 뜬다. 

$ wget http://localhost:8090
--2019-07-29 04:19:04--  http://localhost:8090/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8090... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://localhost:8090/login.action?os_destination=%2Findex.action&permissionViolation=true [following]
--2019-07-29 04:19:04--  http://localhost:8090/login.action?os_destination=%2Findex.action&permissionViolation=true
Reusing existing connection to localhost:8090.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html

index.html                                     [ <=>                                                                                    ]  19.64K  --.-KB/s   in 0.002s 

2019-07-29 04:19:04 (11.1 MB/s) - ‘index.html’ saved [20107]

 

잘된다.

반응형

'기타' 카테고리의 다른 글

TCP 통계  (445) 2020.07.14
Kerberos Setup (KR)  (1401) 2019.08.22
squid proxy  (8) 2018.12.19
reverse proxy 개념  (10) 2018.12.19
Subnet mask 개념  (10) 2018.11.14

+ Recent posts