亚洲精品久久久激情综合-亚洲深深色噜噜狠狠爱网站

07-11

2017

基于Maven+SSM整合shiro+Redis實現(xiàn)后臺管理項目

本項目由賣咸魚叔叔開發(fā)完成，歡迎大神指點，慎重抄襲！參考了sojson提供的demo，和官方文檔介紹。完整實現(xiàn)了用戶、角色、權(quán)限CRUD及分頁，還有shiro的登錄認(rèn)證+授權(quán)訪問控制。項目架構(gòu)：Maven + SpringMVC + Spring + Mybatis + Shiro + Redis數(shù)據(jù)庫：MySql前端框架：H-ui 首先創(chuàng)建Maven項目1.pom.xml 加入依賴包<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>sys</groupId> <artifactId>sys</artifactId> <packaging>war</packaging> <version>0.0.1-SNAPSHOT</version> <name>sys Maven Webapp</name> <url>http://maven.apache.org</url> <dependencies>   <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context-support</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-orm</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-webmvc</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-aspects</artifactId> <version>4.3.2.RELEASE</version> </dependency>  <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.6.0</version> </dependency>   <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis</artifactId> <version>3.3.0</version> </dependency>  <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis-spring</artifactId> <version>1.2.3</version> </dependency>  <dependency> <groupId>com.github.pagehelper</groupId> <artifactId>pagehelper</artifactId> <version>4.1.6</version> </dependency>  <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.37</version> </dependency>  <dependency> <groupId>com.mchange</groupId> <artifactId>c3p0</artifactId> <version>0.9.5</version> </dependency>  <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> <version>2.9.0</version> </dependency>  <dependency> <groupId>commons-fileupload</groupId> <artifactId>commons-fileupload</artifactId> <version>1.3.1</version> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.5</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-email</artifactId> <version>1.4</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.7</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.2</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpcore</artifactId> <version>4.4.4</version> </dependency>  <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> </dependency>  <dependency> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> <version>3.1.0</version> <scope>provided</scope> </dependency>  <dependency> <groupId>javax.servlet.jsp</groupId> <artifactId>jsp-api</artifactId> <version>2.2</version> <scope>provided</scope> </dependency>  <dependency> <groupId>org.glassfish.web</groupId> <artifactId>jstl-impl</artifactId> <version>1.2</version> <exclusions> <exclusion> <groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId> </exclusion> <exclusion> <groupId>javax.servlet.jsp</groupId> <artifactId>jsp-api</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>net.sf.json-lib</groupId> <artifactId>json-lib</artifactId> <version>2.4</version> <classifier>jdk15</classifier> </dependency> <dependency> <groupId>commons-httpclient</groupId> <artifactId>commons-httpclient</artifactId> <version>3.1</version> </dependency>  <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-core</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-spring</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-cas</artifactId> <version>1.2.3</version> <exclusions> <exclusion> <groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-web</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-ehcache</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.apache.shiro</groupId> <artifactId>shiro-quartz</artifactId> <version>1.2.3</version> </dependency>  </dependencies> <build> <finalName>sys</finalName> <plugins>  <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> </plugins> </build></project>2.SSM框架配置beans.xml<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:context="http://www.springframework.org/schema/context" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd">  <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"> <property name="driverClass" value="com.mysql.jdbc.Driver"/> <property name="jdbcUrl" value="jdbc:mysql://localhost:3306/sys_test?characterEncoding=UTF8&allowMultiQueries=true"/> <property name="user" value="root"/> <property name="password" value="root"/> <property name="maxPoolSize" value="100"/> <property name="minPoolSize" value="10"/> <property name="maxIdleTime" value="60"/> </bean>  <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean"> <property name="dataSource" ref="dataSource"/> <property name="configLocation" value="classpath:mybatis-config.xml"></property> </bean>  <bean class="org.mybatis.spring.mapper.MapperScannerConfigurer"> <property name="basePackage" value="com.sys.mapper" /> </bean>  <bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="dataSource" /> </bean>  <tx:advice id="txAdvice" transaction-manager="txManager"> <tx:attributes> <tx:method name="select*" propagation="SUPPORTS" read-only="true"/> <tx:method name="page*" propagation="SUPPORTS" read-only="true"/> <tx:method name="is*" propagation="SUPPORTS" read-only="true"/> <tx:method name="*" propagation="REQUIRED" read-only="false"/> </tx:attributes> </tx:advice>  <aop:config> <aop:pointcut id="txPointcut" expression="execution(* com.sys.service..*.*(..))" /> <aop:advisor advice-ref="txAdvice" pointcut-ref="txPointcut" /> </aop:config>   <context:component-scan base-package="com.sys.service"></context:component-scan>  <aop:aspectj-autoproxy/> </beans>dispatcher-servlet.xml<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:context="http://www.springframework.org/schema/context" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd">  <context:component-scan base-package="com.sys" />  <mvc:annotation-driven />  <aop:aspectj-autoproxy/>  <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="prefix" value="/WEB-INF/jsp/"></property> <property name="suffix" value=".jsp"></property> </bean>  <bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver"> <property name="maxUploadSize" value="104857600" /> <property name="defaultEncoding" value="UTF-8" /> <property name="maxInMemorySize" value="40960" /> </bean>  <mvc:resources location="/css/" mapping="/css/**" /> <mvc:resources location="/js/" mapping="/js/**" /> <mvc:resources location="/images/" mapping="/images/**" /> <mvc:resources location="/skin/" mapping="/skin/**" /> <mvc:resources location="/lib/" mapping="/lib/**" /> </beans>mybatis-config.xml<?xml version="1.0" encoding="UTF-8" ?><!DOCTYPE configuration PUBLIC "-//mybatis.org//DTD Config 3.0//EN" "http://mybatis.org/dtd/mybatis-3-config.dtd"><configuration> <settings>  <setting name="logImpl" value="LOG4J2"/> </settings> <typeAliases>  <package name="com.sys.pojo"/> </typeAliases> <mappers>  <package name="com.sys.mapper"/> </mappers></configuration>Log4j2.xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE xml><Configuration status="OFF"> <Appenders> <Console name="CONSOLE" target="SYSTEM_OUT"> <PatternLayout pattern="%d{HH:mm:ss.SSS} %-5level %logger{0} - %msg%n" /> </Console> <RollingFile name="ROLLING" fileName="/logs/ups-manager/log.log" filePattern="/logs/log_%d{yyyy-MM-dd}_%i.log"> <PatternLayout pattern="%d %p %c{1.} [%t] %m%n"/> <Policies> <TimeBasedTriggeringPolicy modulate="true" interval="1"/> <SizeBasedTriggeringPolicy size="1024 KB"/> </Policies> <DefaultRolloverStrategy max="100"/> </RollingFile> </Appenders> <Loggers> <Root level="debug"> <AppenderRef ref="CONSOLE" /> <AppenderRef ref="ROLLING"/> </Root>  <Logger name="org.mybatis.spring" level="error"> <AppenderRef ref="CONSOLE"/> </Logger> </Loggers></Configuration>web.xml<?xml version="1.0" encoding="UTF-8"?><web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd" id="WebApp_ID" version="3.1">  <listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener>  <filter> <filter-name>characterEncodingFilter</filter-name> <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class> <init-param> <param-name>encoding</param-name> <param-value>UTF-8</param-value> </init-param> <init-param>  <param-name>forceEncoding</param-name> <param-value>true</param-value> </init-param> </filter> <filter-mapping> <filter-name>characterEncodingFilter</filter-name>  <url-pattern>/*</url-pattern> </filter-mapping> <context-param> <param-name>contextConfigLocation</param-name> <param-value> classpath:spring-shiro.xml, classpath:beans.xml, classpath:dispatcher-servlet.xml </param-value> </context-param>  <filter> <filter-name>shiroFilter</filter-name> <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class> <init-param> <param-name>targetFilterLifecycle</param-name> <param-value>true</param-value> </init-param> </filter> <filter-mapping> <filter-name>shiroFilter</filter-name> <url-pattern>/*</url-pattern> </filter-mapping>  <servlet> <servlet-name>dispatcher</servlet-name> <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> <init-param> <param-name>contextConfigLocation</param-name> <param-value>classpath:dispatcher-servlet.xml</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>dispatcher</servlet-name> <url-pattern>/</url-pattern> </servlet-mapping> </web-app>3.實現(xiàn)用戶、角色、權(quán)限頁面操作功能，這里就不貼代碼了4.接下來就是shiro，登錄認(rèn)證和授權(quán)只需要繼承AuthorizingRealm，其繼承了 AuthenticatingRealm（即身份驗證），而且也間接繼承了 CachingRealm（帶有緩存實現(xiàn)）。身份認(rèn)證重寫doGetAuthenticationInfo方法，授權(quán)重寫doGetAuthorizationInfo方法。Shiro 默認(rèn)提供的 Realm 身份認(rèn)證流程流程如下：首先調(diào)用 Subject.login(token) 進(jìn)行登錄，其會自動委托給 Security Manager，調(diào)用之前必須通過 SecurityUtils.setSecurityManager() 設(shè)置；SecurityManager 負(fù)責(zé)真正的身份驗證邏輯；它會委托給 Authenticator 進(jìn)行身份驗證；Authenticator 才是真正的身份驗證者，Shiro API 中核心的身份認(rèn)證入口點，此處可以自定義插入自己的實現(xiàn)；Authenticator 可能會委托給相應(yīng)的 AuthenticationStrategy 進(jìn)行多 Realm 身份驗證，默認(rèn) ModularRealmAuthenticator 會調(diào)用 AuthenticationStrategy 進(jìn)行多 Realm 身份驗證；Authenticator 會把相應(yīng)的 token 傳入 Realm，從 Realm 獲取身份驗證信息，如果沒有返回 / 拋出異常表示身份驗證失敗了。此處可以配置多個 Realm，將按照相應(yīng)的順序及策略進(jìn)行訪問。代碼實現(xiàn)：/** * */package com.sys.shiro;import javax.annotation.Resource;import org.apache.shiro.SecurityUtils;import org.apache.shiro.authc.AccountException;import org.apache.shiro.authc.AuthenticationInfo;import org.apache.shiro.authc.AuthenticationToken;import org.apache.shiro.authc.DisabledAccountException;import org.apache.shiro.authc.SimpleAuthenticationInfo;import org.apache.shiro.authz.AuthorizationInfo;import org.apache.shiro.authz.SimpleAuthorizationInfo;import org.apache.shiro.realm.AuthorizingRealm;import org.apache.shiro.subject.PrincipalCollection;import org.apache.shiro.subject.SimplePrincipalCollection;import org.apache.shiro.util.ByteSource;import com.sys.pojo.AdminUser;import com.sys.service.AdminUserService;/** * @ClassName: MyRealm * @Description: shiro 認(rèn)證 + 授權(quán) 重寫 */public class MyRealm extends AuthorizingRealm { @Resource AdminUserService adminUserService; /* (non-Javadoc) * @see org.apache.shiro.realm.AuthorizingRealm#doGetAuthorizationInfo(org.apache.shiro.subject.PrincipalCollection) */ /** * 授權(quán)Realm */ @Override protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principals) { String account = (String)principals.getPrimaryPrincipal(); AdminUser pojo = new AdminUser(); pojo.setAccount(account); Long userId = adminUserService.selectOne(pojo).getId(); SimpleAuthorizationInfo info = new SimpleAuthorizationInfo(); /**根據(jù)用戶ID查詢角色（role），放入到Authorization里.*/ info.setRoles(adminUserService.findRoleByUserId(userId)); /**根據(jù)用戶ID查詢權(quán)限（permission），放入到Authorization里.*/ info.setStringPermissions(adminUserService.findPermissionByUserId(userId)); return info; } /* (non-Javadoc) * @see org.apache.shiro.realm.AuthenticatingRealm#doGetAuthenticationInfo(org.apache.shiro.authc.AuthenticationToken) */ /** * 登錄認(rèn)證Realm */ @Override protected AuthenticationInfo doGetAuthenticationInfo(AuthenticationToken token) { String username = (String)token.getPrincipal(); String password = new String((char[])token.getCredentials()); AdminUser user = adminUserService.login(username, password); if(null==user){ throw new AccountException("帳號或密碼不正確！"); } if(user.getIsDisabled()){ throw new DisabledAccountException("帳號已經(jīng)禁止登錄！"); } //**密碼加鹽**交給AuthenticatingRealm使用CredentialsMatcher進(jìn)行密碼匹配 return new SimpleAuthenticationInfo(user.getAccount(),user.getPassword(),ByteSource.Util.bytes("3.14159"), getName()); } /** * 清空當(dāng)前用戶權(quán)限信息 */ public void clearCachedAuthorizationInfo() { PrincipalCollection principalCollection = SecurityUtils.getSubject().getPrincipals(); SimplePrincipalCollection principals = new SimplePrincipalCollection( principalCollection, getName()); super.clearCachedAuthorizationInfo(principals); } /** * 指定principalCollection 清除 */ public void clearCachedAuthorizationInfo(PrincipalCollection principalCollection) { SimplePrincipalCollection principals = new SimplePrincipalCollection( principalCollection, getName()); super.clearCachedAuthorizationInfo(principals); } }shiro的配置：spring-shiro.xml<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:util="http://www.springframework.org/schema/util" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd"> <bean id="shiroFilter" class="org.apache.shiro.spring.web.ShiroFilterFactoryBean">  <property name="securityManager" ref="securityManager"></property>  <property name="loginUrl" value="/login"></property>  <property name="unauthorizedUrl" value="/unauthorized.jsp"></property>  <property name="filters"> <map>  <entry key="kickout" value-ref="KickoutSessionControlFilter"/> </map> </property>   <property name="filterChainDefinitions"> <value> /js/**=anon /css/**=anon /images/**=anon /skin/**=anon /lib/**=anon /nodel/**=anon /WEB-INF/jsp/**=anon /adminUserLogin/**=anon /**/submitLogin.do=anon /**=user,kickout </value> </property> </bean>  <bean id="lifecycleBeanPostProcessor" class="org.apache.shiro.spring.LifecycleBeanPostProcessor" />  <bean id="securityManager" class="org.apache.shiro.web.mgt.DefaultWebSecurityManager"> <property name="realm" ref="MyRealm"/> <property name="rememberMeManager" ref="rememberMeManager"/> </bean> <bean id="MyRealm" class="com.sys.shiro.MyRealm" > <property name="cachingEnabled" value="false"/> </bean>  <bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean"> <property name="staticMethod" value="org.apache.shiro.SecurityUtils.setSecurityManager"/> <property name="arguments" ref="securityManager"/> </bean>  <bean id="sessionIdCookie" class="org.apache.shiro.web.servlet.SimpleCookie"> <constructor-arg value="rememberMe"/> <property name="httpOnly" value="true"/> <property name="maxAge" value="-1"/> </bean>  <bean id="rememberMeCookie" class="org.apache.shiro.web.servlet.SimpleCookie"> <constructor-arg value="rememberMe"/> <property name="httpOnly" value="true"/>  <property name="maxAge" value="604800"/> </bean>  <bean id="rememberMeManager" class="org.apache.shiro.web.mgt.CookieRememberMeManager">  <property name="cipherKey" value="#{T(org.apache.shiro.codec.Base64).decode('3AvVhmFLUs0KTA3Kprsdag==')}"/> <property name="cookie" ref="rememberMeCookie"/> </bean>  <bean id="RememberMeFilter" class="com.sys.shiro.RememberMeFilter" />    <bean id="KickoutSessionControlFilter" class="com.sys.shiro.KickoutSessionControlFilter"> </bean> </beans>5.登錄即密碼失敗多次后鎖定/** * */package com.sys.controller;import java.util.LinkedHashMap;import java.util.Map;import javax.annotation.Resource;import javax.servlet.http.HttpServletRequest;import org.apache.logging.log4j.LogManager;import org.apache.logging.log4j.Logger;import org.apache.shiro.SecurityUtils;import org.apache.shiro.authc.AccountException;import org.apache.shiro.authc.AuthenticationException;import org.apache.shiro.authc.DisabledAccountException;import org.apache.shiro.authc.ExcessiveAttemptsException;import org.apache.shiro.authc.UsernamePasswordToken;import org.apache.shiro.subject.Subject;import org.springframework.stereotype.Controller;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestMethod;import org.springframework.web.bind.annotation.ResponseBody;import com.sys.pojo.AdminUser;import com.sys.service.AdminUserService;import com.sys.common.JedisUtils;import com.sys.shiro.ShiroUtils;/** * @ClassName: LoginController * @Description: 登錄*/@Controllerpublic class LoginController{ protected final static Logger logger = LogManager.getLogger(LoginController.class); protected Map<String, Object> resultMap = new LinkedHashMap<String, Object>(); @Resource AdminUserService adminUserService; /** * @Description: 登錄認(rèn)證 * @param um 登錄賬號 * @param pw 登錄密碼 * @param rememberMe 記住我 * @param request * @return * @throws * @author lao * @Date 2018年1月15日下午12:24:19 * @version 1.00 */ @RequestMapping(value="/submitLogin.do",method=RequestMethod.POST) @ResponseBody public Map<String,Object> submitLogin(String um,String pw,boolean rememberMe,HttpServletRequest request){ Subject subject = SecurityUtils.getSubject(); UsernamePasswordToken token = new UsernamePasswordToken(um,ShiroUtils.getStrByMD5(pw)); try{ token.setRememberMe(rememberMe); subject.login(token); JedisUtils.del(um); logger.info("------------------身份認(rèn)證成功-------------------"); resultMap.put("status", 200); resultMap.put("message", "登錄成功！"); } catch (DisabledAccountException dax) { logger.info("用戶名為:" + um + " 用戶已經(jīng)被禁用！"); resultMap.put("status", 500); resultMap.put("message", "帳號已被禁用！"); } catch (ExcessiveAttemptsException eae) { logger.info("用戶名為:" + um + " 用戶登錄次數(shù)過多，有暴力破解的嫌疑！"); resultMap.put("status", 500); resultMap.put("message", "登錄次數(shù)過多！"); } catch (AccountException ae) { logger.info("用戶名為:" + token.getPrincipal() + " 帳號或密碼錯誤！"); String excessiveInfo = ExcessiveAttemptsInfo(um); if(null!=excessiveInfo){ resultMap.put("status", 500); resultMap.p

01-17

2018

解Bug之路：記一次JVM堆外內(nèi)存泄露Bug的查找

前言JVM的堆外內(nèi)存泄露的定位一直是個比較棘手的問題。此次的Bug查找從堆內(nèi)內(nèi)存的泄露反推出堆外內(nèi)存，同時對物理內(nèi)存的使用做了定量的分析，從而實錘了Bug的源頭。筆者將此Bug分析的過程寫成博客，以饗讀者。由于物理內(nèi)存定量分析部分用到了linux kernel虛擬內(nèi)存管理的知識，讀者如果有興趣了解請看ulk3(《深入理解linux內(nèi)核第三版》)內(nèi)存泄露Bug現(xiàn)場一個線上穩(wěn)定運行了三年的系統(tǒng)，從物理機(jī)遷移到docker環(huán)境后，運行了一段時間，突然被監(jiān)控系統(tǒng)發(fā)出了某些實例不可用的報警。所幸有負(fù)載均衡，可以自動下掉節(jié)點，如下圖所示:登錄到對應(yīng)機(jī)器上后，發(fā)現(xiàn)由于內(nèi)存占用太大，觸發(fā)OOM，然后被linux系統(tǒng)本身給kill了。應(yīng)急措施緊急在出問題的實例上再次啟動應(yīng)用，啟動后，內(nèi)存占用正常，一切Okay。奇怪現(xiàn)象當(dāng)前設(shè)置的最大堆內(nèi)存是1792M，如下所示:-Xmx1792m -Xms1792m -Xmn900m -XX:PermSi ze=256m -XX:MaxPermSize=256m -server -Xss512k查看操作系統(tǒng)層面的監(jiān)控，發(fā)現(xiàn)內(nèi)存占用情況如下圖所示:上圖藍(lán)色的線表示總的內(nèi)存使用量，發(fā)現(xiàn)一直漲到了4G后，超出了系統(tǒng)限制。很明顯，有堆外內(nèi)存泄露了。查找線索gc日志一般出現(xiàn)內(nèi)存泄露，筆者立馬想到的就是查看當(dāng)時的gc日志。本身應(yīng)用所采用框架會定時打印出對應(yīng)的gc日志，遂查看，發(fā)現(xiàn)gc日志一切正常。對應(yīng)日志如下:查看了當(dāng)天的所有g(shù)c日志，發(fā)現(xiàn)內(nèi)存始終會回落到170M左右，并無明顯的增加。要知道JVM進(jìn)程本身占用的內(nèi)存可是接近4G(加上其它進(jìn)程,例如日志進(jìn)程就已經(jīng)到4G了)，進(jìn)一步確認(rèn)是堆外內(nèi)存導(dǎo)致。排查代碼打開線上服務(wù)對應(yīng)對應(yīng)代碼，查了一圈，發(fā)現(xiàn)沒有任何地方顯式利用堆外內(nèi)存，其沒有依賴任何額外的native方法。關(guān)于網(wǎng)絡(luò)IO的代碼也是托管給Tomcat，很明顯，作為一個全世界廣泛流行的Web服務(wù)器，Tomcat不大可能有堆外內(nèi)存泄露。進(jìn)一步查找由于在代碼層面沒有發(fā)現(xiàn)堆外內(nèi)存的痕跡，那就繼續(xù)找些其它的信息，希望能發(fā)現(xiàn)蛛絲馬跡。Dump出JVM的Heap堆由于線上出問題的Server已經(jīng)被kill，還好有其它幾臺，登上去發(fā)現(xiàn)它們也占用了很大的堆外內(nèi)存，只是還沒有到觸發(fā)OOM的臨界點而已。于是就趕緊用jmap dump了兩臺機(jī)器中應(yīng)用JVM的堆情況，這兩臺留做現(xiàn)場保留不動，然后將其它機(jī)器迅速重啟，以防同時被OOM導(dǎo)致服務(wù)不可用。使用如下命令dump:jmap -dump:format=b，file=heap.bin [pid]使用MAT分析Heap文件挑了一個heap文件進(jìn)行分析，堆的使用情況如下圖所示:一共用了200多M，和之前gc文件打印出來的170M相差不大，遠(yuǎn)遠(yuǎn)沒有到4G的程度。不得不說MAT是個非常好用的工具，它可以提示你可能內(nèi)存泄露的點:這個cachedBnsClient類有12452個實例，占用了整個堆的61.92%。查看了另一個heap文件，發(fā)現(xiàn)也是同樣的情況。這個地方肯定有內(nèi)存泄露，但是也占用了130多M，和4G相差甚遠(yuǎn)。查看對應(yīng)的代碼系統(tǒng)中大部分對于CachedBnsClient的調(diào)用，都是通過注解Autowired的，這部分實例數(shù)很少。唯一頻繁產(chǎn)生此類實例的代碼如下所示:@Override public void fun() { BnsClient bnsClient = new CachedBnsClient(); // do something return ; }此CachedBnsClient僅僅在方法體內(nèi)使用，并沒有逃逸到外面，再看此類本身public class CachedBnsClient { private ConcurrentHashMap<String， List<String>> authCache = new ConcurrentHashMap<String， List<String>>(); private ConcurrentHashMap<String， List<URI>> validUriCache = new ConcurrentHashMap<String， List<URI>>(); private ConcurrentHashMap<String， List<URI>> uriCache = new ConcurrentHashMap<String， List<URI>>(); ...... }沒有任何static變量，同時也沒有往任何全局變量注冊自身。換言之，在類的成員(Member)中，是不可能出現(xiàn)內(nèi)存泄露的。當(dāng)時只粗略的過了一過成員變量，回過頭來細(xì)想，還是漏了不少地方的。更多信息由于代碼排查下來，感覺這塊不應(yīng)該出現(xiàn)內(nèi)存泄露(但是事實確是如此的打臉)。這個類也沒有顯式用到堆外內(nèi)存，而且只占了130M，和4G比起來微不足道，還是先去追查主要矛盾再說。使用jstack dump線程信息現(xiàn)場信息越多，越能找出蛛絲馬跡。先用jstack把線程信息dump下來看下。這一看，立馬發(fā)現(xiàn)了不同，除了正常的IO線程以及框架本身的一些守護(hù)線程外，竟然還多出來了12563多個線程。"Thread-5" daemon prio=10 tid=0x00007fb79426e000 nid=0x7346 waiting on condition [0x00007fb7b5678000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at com.xxxxx.CachedBnsClient$1.run(CachedBnsClient.java:62)而且這些正好是運行再CachedBnsClient的run方法上面！這些特定線程的數(shù)量正好是12452個，和cachedBnsClient數(shù)量一致!再次check對應(yīng)代碼原來剛才看CachedBnsClient代碼的時候遺漏掉了一個關(guān)鍵的點! public CachedBnsClient(BnsClient client) { super(); this.backendClient = client; new Thread() { @Override public void run() { for (; ; ) { refreshCache(); try { Thread.sleep(60 * 1000); } catch (InterruptedException e) { logger.error("出錯"， e); } } } }這段代碼是CachedBnsClient的構(gòu)造函數(shù)，其在里面創(chuàng)建了一個無限循環(huán)的線程，每隔60s啟動一次刷新一下里面的緩存!找到關(guān)鍵點在看到12452個等待在CachedBnsClient.run的業(yè)務(wù)的一瞬間筆者就意識到，肯定是這邊的線程導(dǎo)致對外內(nèi)存泄露了。下面就是根據(jù)線程大小計算其泄露內(nèi)存量是不是確實能夠引起OOM了。發(fā)現(xiàn)內(nèi)存計算對不上由于我們這邊設(shè)置的Xss是512K，即一個線程棧大小是512K，而由于線程共享其它MM單元(線程本地內(nèi)存是是現(xiàn)在線程棧上的)，所以實際線程堆外內(nèi)存占用數(shù)量也是512K。進(jìn)行如下計算:12563 * 512K = 6331M = 6.3G整個環(huán)境一共4G，加上JVM堆內(nèi)存1.8G(1792M)，已經(jīng)明顯的超過了4G。(6.3G + 1.8G)=8.1G > 4G如果按照此計算，應(yīng)用應(yīng)用早就被OOM了。怎么回事呢？為了解決這個問題，筆者又思考了好久。如下所示:Java線程底層實現(xiàn)JVM的線程在linux上底層是調(diào)用NPTL(Native Posix Thread Library)來創(chuàng)建的，一個JVM線程就對應(yīng)linux的lwp(輕量級進(jìn)程，也是進(jìn)程，只不過共享了mm_struct，用來實現(xiàn)線程)，一個thread.start就相當(dāng)于do_fork了一把。其中，我們在JVM啟動時候設(shè)置了-Xss=512K(即線程棧大小)，這512K中然后有8K是必須使用的，這8K是由進(jìn)程的內(nèi)核棧和thread_info公用的，放在兩塊連續(xù)的物理頁框上。如下圖所示:眾所周知，一個進(jìn)程(包括lwp)包括內(nèi)核棧和用戶棧，內(nèi)核棧+thread_info用了8K，那么用戶態(tài)的?？捎脙?nèi)存就是:512K-8K=504K如下圖所示:Linux實際物理內(nèi)存映射事實上linux對物理內(nèi)存的使用非常的摳門，一開始只是分配了虛擬內(nèi)存的線性區(qū)，并沒有分配實際的物理內(nèi)存，只有推到最后使用的時候才分配具體的物理內(nèi)存，即所謂的請求調(diào)頁。如下圖所示:查看smaps進(jìn)程內(nèi)存使用信息使用如下命令，查看cat /proc/[pid]/smaps > smaps.txt實際物理內(nèi)存使用信息，如下所示:7fa69a6d1000-7fa69a74f000 rwxp 00000000 00:00 0 Size: 504 kB Rss: 92 kB Pss: 92 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 92 kB Referenced: 92 kB Anonymous: 92 kB AnonHugePages: 0 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB 7fa69a7d3000-7fa69a851000 rwxp 00000000 00:00 0 Size: 504 kB Rss: 152 kB Pss: 152 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 152 kB Referenced: 152 kB Anonymous: 152 kB AnonHugePages: 0 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB搜索下504KB，正好是12563個，對了12563個線程，其中Rss表示實際物理內(nèi)存(含共享庫)92KB，Pss表示實際物理內(nèi)存(按比例共享庫)92KB(由于沒有共享庫，所以Rss==Pss)，以第一個7fa69a6d1000-7fa69a74f000線性區(qū)來看，其映射了92KB的空間，第二個映射了152KB的空間。如下圖所示:挑出符合條件（即size是504K）的幾十組看了下，基本都在92K-152K之間，再加上內(nèi)核棧8K(92+152)/2+8K=130K，由于是估算，取整為128K，即反映此應(yīng)用平均線程棧大小。注意，實際內(nèi)存有波動的原因是由于環(huán)境不同，從而走了不同的分支，導(dǎo)致棧上的增長不同。重新進(jìn)行內(nèi)存計算JVM一開始申請了-Xmx1792m -Xms1792m即1.8G的堆內(nèi)內(nèi)存，這里是即時分配，一開始就用物理頁框填充。12563個線程，每個線程棧平均大小128K，即:128K * 12563=1570M=1.5G的對外內(nèi)存取個整數(shù)128K，就能反映出平均水平。再拿這個128K * 12563 =1570M = 1.5G，加上JVM的1.8G，就已經(jīng)達(dá)到了3.3G，再加上kernel和日志傳輸進(jìn)程等使用的內(nèi)存數(shù)量，確實已經(jīng)接近了4G，這樣內(nèi)存就對應(yīng)上了！(注:用于定量內(nèi)存計算的環(huán)境是一臺內(nèi)存用量將近4G，但還沒OOM的機(jī)器)為什么在物理機(jī)上沒有應(yīng)用Down機(jī)筆者登錄了原來物理機(jī)，應(yīng)用還在跑，發(fā)現(xiàn)其同樣有堆外內(nèi)存泄露的現(xiàn)象，其物理內(nèi)存使用已經(jīng)達(dá)到了5個多G!幸好物理機(jī)內(nèi)存很大，而且此應(yīng)用發(fā)布還比較頻繁，所以沒有被OOM。Dump了物理機(jī)上應(yīng)用的線程，一共有28737個線程，其中28626個線程等待在CachedBnsClient上。同樣用smaps查看進(jìn)程實際內(nèi)存信息，其平均大小依舊為128K，因為是同一應(yīng)用的原因繼續(xù)進(jìn)行物理內(nèi)存計算1.8+(28737 * 128k)/1024K =(3.6+1.8)=5.4G進(jìn)一步驗證了我們的推理。這么多線程應(yīng)用為什么沒有卡頓因為基本所有的線程都睡眠在 Thread.sleep(60 * 1000);//一次睡眠60s上。所以僅僅占用了內(nèi)存，實際占用的CPU時間很少?？偨Y(jié)查找Bug的時候，現(xiàn)場信息越多越好，同時定位Bug必須要有實質(zhì)性的證據(jù)。例如內(nèi)存泄露就要用你推測出的模型進(jìn)行定量分析。在定量和實際對不上的時候，深挖下去，你會發(fā)現(xiàn)不一樣的風(fēng)景!

辽阳剐什教育咨询有限公司

技術(shù)學(xué)院