redis水平扩展实践，完全配置，无需代码改动

设计思路

思路很简单，就是基于用户ID进行分库，将用户的ID字符串按照byte逐个计算ID对应的hash原值（一个数字，取绝对值，因为原始值可能过大溢出，变成负数），然后，再用这个hash原值对库的个数进行求模，这个模值就是库列表的索引值，也就选择好了用什么库。

 

hash算法

 1 /** 2  * Created by chengsh05 on 2017/12/22. 3  */ 4 public class BKDRHashUtil { 5     public static int BKDRHash(char[] str) { 6         int seed = 131; // 31 131 1313 13131 131313 etc.. 7         int hash = 0; 8         for (int i = 0; i < str.length; i++) { 9             hash = hash * seed + (str[i]);10         }11         return (hash & 0x7FFFFFFF);12     }13 }

 

对于redis的水平扩容，做到完全基于配置实现扩容，最好选择通过xml的配置的方式实现，因为配置在线上只需要改改配置信息，既可以重启服务器实现更新扩容。其实，其他中间件的扩容一样可以这么个逻辑实现。

XML配置

 1 <?xml version="1.0" encoding="UTF-8"?> 2 <beans xmlns="http://www.springframework.org/schema/beans" 3        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 4        xmlns:context="http://www.springframework.org/schema/context" 5        xsi:schemaLocation="http://www.springframework.org/schema/beans 6                           http://www.springframework.org/schema/beans/spring-beans-4.3.xsd 7                           http://www.springframework.org/schema/context 8                           http://www.springframework.org/schema/context/spring-context-4.3.xsd"> 9 10     <bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">11         <property name="maxIdle" value="${spring.redis.pool.maxIdle}"></property>12         <property name="minIdle" value="${spring.redis.pool.minIdle}"></property>13         <property name="maxTotal" value="${spring.redis.pool.maxActive}"></property>14         <property name="maxWaitMillis" value="${spring.redis.pool.maxWait}"></property>15     </bean>16 17     <!--第一组主从redis-->18     <bean id="jedisConnectionFactoryMaster1" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory" destroy-method="destroy" primary="true">19         <property name="poolConfig" ref="jedisPoolConfig"></property>20         <property name="hostName" value="${spring.master1.redis.hostName}"></property>21         <property name="port" value="${spring.master1.redis.port}"></property>22         <property name="database" value="${spring.redis.database}"></property>23         <property name="timeout" value="${spring.redis.timeout}"></property>24     </bean>25     <bean id="redisTemplateMaster1" class="org.springframework.data.redis.core.RedisTemplate">26         <property name="connectionFactory" ref="jedisConnectionFactoryMaster1"></property>27     </bean>28     <bean id="jedisConnectionFactorySlave1" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory" destroy-method="destroy">29         <property name="poolConfig" ref="jedisPoolConfig"></property>30         <property name="hostName" value="${spring.slave1.redis.hostName}"></property>31         <property name="port" value="${spring.slave1.redis.port}"></property>32         <property name="database" value="${spring.redis.database}"></property>33         <property name="timeout" value="${spring.redis.timeout}"></property>34     </bean>35     <bean id="redisTemplateSlave1" class="org.springframework.data.redis.core.RedisTemplate">36         <property name="connectionFactory" ref="jedisConnectionFactorySlave1"></property>37     </bean>38 39     <!-- 第2组主从redis -->40     <bean id="jedisConnectionFactoryMaster2" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory" destroy-method="destroy">41         <property name="poolConfig" ref="jedisPoolConfig"></property>42         <property name="hostName" value="${spring.master2.redis.hostName}"></property>43         <property name="port" value="${spring.master2.redis.port}"></property>44         <property name="database" value="${spring.redis.database}"></property>45         <property name="timeout" value="${spring.redis.timeout}"></property>46     </bean>47     <bean id="redisTemplateMaster2" class="org.springframework.data.redis.core.RedisTemplate">48         <property name="connectionFactory" ref="jedisConnectionFactoryMaster2"></property>49     </bean>50     <bean id="jedisConnectionFactorySlave2" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory" destroy-method="destroy">51         <property name="poolConfig" ref="jedisPoolConfig"></property>52         <property name="hostName" value="${spring.slave2.redis.hostName}"></property>53         <property name="port" value="${spring.slave2.redis.port}"></property>54         <property name="database" value="${spring.redis.database}"></property>55         <property name="timeout" value="${spring.redis.timeout}"></property>56     </bean>57     <bean id="redisTemplateSlave2" class="org.springframework.data.redis.core.RedisTemplate">58         <property name="connectionFactory" ref="jedisConnectionFactorySlave2"></property>59     </bean>60 61     <bean id="commonRedisService" class="org.whuims.web.service.AutoScaleRedisService">62         <!--63               Modified: shihuc, 2017-12-2164                         shihuc, 2018-01-02 全配置，无需计算中间参数，目的就是提升性能。65               注意，这里的改造，是方便redis的水平扩展，目的是为了在增加redis主从服务器的时候，只需要修改一下此处的配置文件，然后重启应用即可。66                  这里配置相对多了点，目的是换取性能。67               另外:1. readRedisTemplateKeyInstancePairs,writeRedisTemplateKeyInstancePairs两个主要的键值结构配置读写实例表。68                   2. 读写redis，主从关系，必须配对填写好，不要出现主从的错位配置。例如rw1、rr1表示第一组的写读关系。69                   3. readRedisKeys列表取值必须和readRedisTemplateKeyInstancePairs的key值一样，writeRedisKeys列表的取值必须70                   和writeRedisTemplateKeyInstancePairs的key值一样。71         -->72         <property name="readRedisTemplateKeyInstancePairs">73             <map key-type="java.lang.String">74                 <entry key="rr1" value-ref="redisTemplateSlave1"></entry>75                 <entry key="rr2" value-ref="redisTemplateSlave2"></entry>76             </map>77         </property>78         <property name="readRedisKeys">79             <list>80                 <value>rr1</value>81                 <value>rr2</value>82             </list>83         </property>84         <property name="writeRedisTemplateKeyInstancePairs">85             <map key-type="java.lang.String">86                 <entry key="rw1" value-ref="redisTemplateMaster1"></entry>87                 <entry key="rw2" value-ref="redisTemplateMaster2"></entry>88             </map>89         </property>90         <property name="writeRedisKeys">91             <list>92                 <value>rw1</value>93                 <value>rw2</value>94             </list>95         </property>96     </bean>97 </beans>

其中用到的参数，通过spring的占位符逻辑，redis的配置数据来自配置文件，这里配置文件信息，简要示例（springboot的配置文件app.properties里面的局部）：

#common part used in redis configuration for below multi redisspring.redis.pool.maxActive=100spring.redis.pool.maxWait=-1spring.redis.pool.maxIdle=8spring.redis.pool.minIdle=0spring.redis.timeout=0spring.redis.database=3#how many redis group to use is depended your business. but, at least one master/slave configuration needed#below is for master1 redis configurationspring.master1.redis.hostName=100.126.22.177spring.master1.redis.port=6379#below is for slave1 redis configurationspring.slave1.redis.hostName=100.126.22.178spring.slave1.redis.port=6379#below is for master2 redis configurationspring.master2.redis.hostName=100.126.22.189spring.master2.redis.port=6379#below is for slave1 redis configurationspring.slave2.redis.hostName=100.126.22.190spring.slave2.redis.port=6379

 

springboot中Java启用配置

/** * Created by chengsh05 on 2017/12/22. * * 通过XML的方式进行redis的配置管理，目的在于方便容量扩缩的时候，只需要进行配置文件的变更即可，这样 * 可以做到容量的水平管理，不需要动业务逻辑代码。 * * 上线的时候，改改配置文件，再重启一下应用即可完成扩缩容。 */@Configuration@ImportResource(value = {"file:${user.dir}/resources/spring-redis.xml"})public class RedisXmlConfig {}

 

分库服务

  1 /**  2  * Created by chengsh05 on 2017/12/22.  3  *  4  * 方便redis组件的水平扩展，扩展的时候，主要改改spring-redis.xml以及app.properties配置文件，不需要动java  5  * 代码，重启应用，即可实现扩容。  6  */  7 public class AutoScaleRedisService {  8   9     Logger logger = Logger.getLogger(AutoScaleRedisService.class); 10  11     /** 12      * Added by shihuc, 2017-12-22 13      * redis水平扩展，中间层抽象逻辑 14      * 15      * Modified by shihuc 2018-01-02 16      * 将redis水平扩展部分，改成完全基于配置，不需要计算，应用层面，对于源的选取，完全就是读的操作，没有计算了，对于计算性能的提升有好处，配置相对麻烦一点。 17      * 18      * Key: rw1,rr1, and so on 19      * value: RedisTemplate instance 20      */ 21     private Map<String, RedisTemplate<String, Object>> readRedisTemplateKeyInstancePairs; 22  23     private Map<String, RedisTemplate<String, Object>> writeRedisTemplateKeyInstancePairs; 24  25     private List<String> readRedisKeys; 26  27     private List<String> writeRedisKeys; 28  29     public Map<String, RedisTemplate<String, Object>> getReadRedisTemplateKeyInstancePairs() { 30         return readRedisTemplateKeyInstancePairs; 31     } 32  33     public void setReadRedisTemplateKeyInstancePairs(Map<String, RedisTemplate<String, Object>> readRedisTemplateKeyInstancePairs) { 34         this.readRedisTemplateKeyInstancePairs = readRedisTemplateKeyInstancePairs; 35     } 36  37     public Map<String, RedisTemplate<String, Object>> getWriteRedisTemplateKeyInstancePairs() { 38         return writeRedisTemplateKeyInstancePairs; 39     } 40  41     public void setWriteRedisTemplateKeyInstancePairs(Map<String, RedisTemplate<String, Object>> writeRedisTemplateKeyInstancePairs) { 42         this.writeRedisTemplateKeyInstancePairs = writeRedisTemplateKeyInstancePairs; 43     } 44  45     public List<String> getReadRedisKeys() { 46         return readRedisKeys; 47     } 48  49     public void setReadRedisKeys(List<String> readRedisKeys) { 50         this.readRedisKeys = readRedisKeys; 51     } 52  53     public List<String> getWriteRedisKeys() { 54         return writeRedisKeys; 55     } 56  57     public void setWriteRedisKeys(List<String> writeRedisKeys) { 58         this.writeRedisKeys = writeRedisKeys; 59     } 60  61     /** 62      * @author shihuc 63      * @param userId 64      * @return 65      */ 66     private String getReadKey(String userId) { 67         int hash = BKDRHashUtil.BKDRHash(userId.toCharArray()); 68         int abs = Math.abs(hash); 69         int idx = abs % getReadRedisCount(); 70         logger.info("userId: " + userId + ", hash: " + hash + ", idx: " + idx); 71         String insKey = getReadRedisKeys().get(idx); 72         return insKey; 73     } 74  75     /** 76      * @author shihuc 77      * @param userId 78      * @return 79      */ 80     private String getWriteKey(String userId) { 81         int hash = BKDRHashUtil.BKDRHash(userId.toCharArray()); 82         int abs = Math.abs(hash); 83         int idx = abs % getWriteRedisCount(); 84         logger.info("userId: " + userId + ", hash: " + hash + ", idx: " + idx); 85         String insKey = getWriteRedisKeys().get(idx); 86         return insKey; 87     } 88  89     /** 90      * @author shihuc 91      * @return the count of read redis instance 92      */ 93     public int getReadRedisCount() { 94         return readRedisKeys.size(); 95     } 96  97     /** 98      * @author shihuc 99      * @return the count of write redis instance100      */101     public int getWriteRedisCount() {102         return writeRedisKeys.size();103     }104 105     /**106      * @author shihuc107      * @param userId108      * @param type109      * @param log110      * @return111      */112     public RedisTemplate<String, Object> getRedisTemplate(String userId, String type, boolean log){113         return getRedisTemplate(userId, type, log, null);114     }115 116     /**117      * 获取redisTemplate实例118      * @author shihuc119      * @param userId120      * @param type121      * @param log122      * @return123      */124     public RedisTemplate<String, Object> getRedisTemplate(String userId, String type, boolean log, String info){125         String insKey = null;126         RedisTemplate<String, Object> redisTemplate = null;127         if(Constants.REDIS_TYPE_READ.equalsIgnoreCase(type)){128             insKey = getReadKey(userId);129             redisTemplate = readRedisTemplateKeyInstancePairs.get(insKey);130         }else {131             insKey = getWriteKey(userId);132             redisTemplate = writeRedisTemplateKeyInstancePairs.get(insKey);133         }134         if (log) {135             if(info != null) {136                 logger.info("userId: " + userId + ", redis: " + insKey + ", type: " + type + ", info: " + info);137             }else{138                 logger.info("userId: " + userId + ", redis: " + insKey + ", type: " + type);139             }140         }141         return redisTemplate;142     }143 144     /**145      * 用于校验配置的时候，读写实例的key值和键值列表的value之间是否是对应的关系。146      */147     @PostConstruct148     public void init() throws Exception {149         int ridx = 0;150         for(Map.Entry<String, RedisTemplate<String, Object>> rele: readRedisTemplateKeyInstancePairs.entrySet()) {151             String rkey = rele.getKey();152             String trkey = readRedisKeys.get(ridx);153             if(!rkey.equals(trkey)){154                 throw new Exception("[read] redis group configuration error, order is not matched");155             }156             ridx++;157         }158         int widx = 0;159         for(Map.Entry<String, RedisTemplate<String, Object>> wele: writeRedisTemplateKeyInstancePairs.entrySet()) {160             String wkey = wele.getKey();161             String twkey = writeRedisKeys.get(widx);162             if(!wkey.equals(twkey)){163                 throw new Exception("[write] redis group configuration error, order is not matched");164             }165             widx++;166         }167     }168 }

 

使用案例

    @RequestMapping("/redischeck")    @ResponseBody    public String redisCheck(@RequestParam(value = "query") String query) {        System.out.println("check：" + query);        int rdc = autoScaleRedisService.getReadRedisCount();        int wtc = autoScaleRedisService.getWriteRedisCount();        RedisTemplate redisTemplate = autoScaleRedisService.getRedisTemplate(query, Constants.REDIS_TYPE_READ, true, "buildSession");        RedisTemplate redisTemplate2 = autoScaleRedisService.getRedisTemplate(query, Constants.REDIS_TYPE_WRITE, true, "buildSession");        return "rdc: " + rdc + ", wtc: " + wtc;    }

 

整个思路和实现过程，其实非常通俗易懂，非常方便的用于各种中间件的场景，当然，若有特殊需求，也无外乎类似的逻辑。

 

若有什么不妥，欢迎探讨，若有更好的巧妙方案，也可以探讨！

 

转载请指明出处，谢谢！欢迎加关注！