摘要：發現環信的根據時間條件拉取歷史消息接口已經停用就做了個通過導出聊天記錄接口保存到數據庫實體的功能分享一下大致思路通過環信的接口把前一天小時的數據壓縮包下載到本地把下載后的文件解壓讀取處理寫入到實體設置一個定時器定時執行通過環信接口拉取數據并

發現環信的根據時間條件拉取歷史消息接口已經停用,就做了個通過導出聊天記錄接口保存到數據庫實體的功能,分享一下.

大致思路:

1.通過環信的接口,把前一天24小時的數據壓縮包下載到本地
2.把下載后的文件解壓讀取處理,寫入到實體
3.設置一個定時器,定時執行.

1.通過環信接口拉取數據,并解壓讀取
環信聊天記錄下載接口

@Service
public class EaseMobService implements IEaseMobService{

    @Autowired
    private IChatMessageService chatMessageService;
    @Override
    public void saveChatMessages() {
        //下載文件保存路徑
        String filePath = "/opt/apache/chatFiles/";
        //未加時間戳的請求地址
        //OrgInfo.ORG_NAME  環信org_name  OrgInfo.APP_NAME 環信app_name
        String requestUrl = "http://a1.easemob.com/"+ OrgInfo.ORG_NAME + "/" + OrgInfo.APP_NAME + "/chatmessages/";
        //獲取前一天內的時間list
        List hourList = DateUtils.getOneDayHourList(DateUtils.getBeforeDayDate(new Date(), 1));
        //環信token 自己寫一個工具類獲取token
        String token = TokenUtil.getAccessToken();
        //獲取下載地址
        for(String hour: hourList){
            try {
                String downloadUrl = HttpUtil.getEasemobChatMessageDownloadUrl(requestUrl + hour, token);
                if(!"fail".equals(downloadUrl)){
                    //下載壓縮文件到指定文件夾
                    String fileName = hour + ".gz";
                    String downLoadResult = HttpUtil.downloadFileByUrls(downloadUrl, fileName,filePath);
                    //下載成功進行解壓文件和讀取文件
                    if("ok".equals(downLoadResult)){
                        //解壓文件
                        String outPutFilePath = unZipFile(filePath + fileName);
                        //讀取文件
                        if(outPutFilePath.length() >0) {
                            String content = readFile2String(outPutFilePath);
                            //處理文本內容,寫入實體
                            if(content.length() > 0) {
                                chatMessageService.handleContent(content);
                            }
                        }
                    }
                }
                //延時執行,環信下載接口有限流
                Thread.sleep(5000);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

    /**
     * 讀取文件內容
    **/
    private String readFile2String(String outPutFilePath) {
        String content = "";
        String encoding = "UTF-8";
        File file = new File(outPutFilePath);
        Long fileLength = file.length();
        byte[] fileContent = new byte[fileLength.intValue()];
        try {
            FileInputStream in = new FileInputStream(file);
            in.read(fileContent);
            in.close();
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
        try {
            content = new String(fileContent, encoding).trim();
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        return content;
    }

    /**
     * 解壓文件并返回解壓后的文件
    **/
    private String unZipFile(String filePath) {
        //解壓gz壓縮包
        String ouPutFile = "";
        try {
            //建立gzip壓縮文件輸入流
            FileInputStream fIn = new FileInputStream(filePath);
            //建立gzip解壓工作流
            GZIPInputStream gzIn = new GZIPInputStream(fIn);
            //建立解壓文件輸出流
            ouPutFile = filePath.substring(0,filePath.lastIndexOf("."));
            FileOutputStream fOut = new FileOutputStream(ouPutFile);
            int num;
            byte[] buf=new byte[1024];

            while ((num = gzIn.read(buf,0,buf.length)) != -1)
            {
                fOut.write(buf,0,num);
            }
            gzIn.close();
            fOut.close();
            fIn.close();
        } catch (Exception e){
            e.printStackTrace();
        }
        return ouPutFile;
    }
}

DateUtils工具類方法

 /**
     * 獲取指定日期的一天小時集合yyyyMMddHH
    **/
    public static List getOneDayHourList(Date date){
        List hourList = new ArrayList();
        SimpleDateFormat fmt = new SimpleDateFormat("yyyyMMdd");
        String dateString = fmt.format(date);
        for(int i = 0; i < 24; i++) {
            String hour = String.valueOf(i);
            if(i < 10){
                hour = "0" + hour;
            }
            hourList.add(dateString + hour);
        }
        return hourList;
    }
/**
     * 獲取指定日期的前N天日期
    **/
public static Date getBeforeDayDate(Date date, int beforeDay)
    {
        Calendar a = Calendar.getInstance();
        a.setTime(date);
        a.add(Calendar.DATE, -beforeDay);
        return a.getTime();
    }

HttpUtil工具類

public class HttpUtil {

   private static Logger log = LoggerFactory.getLogger(HttpUtil.class);

    public static String getEasemobChatMessageDownloadUrl(String getUrl, String token) {
        String downloadUrl = "";
        try {
            URL url = new URL(getUrl);    //把字符串轉換為URL請求地址
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();// 打開連接
            //設置Head參數
            connection.setRequestProperty("Content-Type", " application/json");//設定 請求格式 json，也可以設定xml格式的
            connection.setRequestProperty("Accept-Charset", "utf-8");  //設置編碼語言
            connection.setRequestProperty("Connection", "keep-alive");  //設置連接的狀態
            connection.setRequestProperty("Authorization", token);
            connection.connect();// 連接會話
            // 獲取輸入流
            BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream(), "UTF-8"));
            String line;
            StringBuilder sb = new StringBuilder();
            while ((line = br.readLine()) != null) {// 循環讀取流
                sb.append(line);
            }
            br.close();// 關閉流
            connection.disconnect();// 斷開連接
            //返回結果處理
            JSONArray jsonArray = JSON.parseArray("[" + sb.toString() + "]");
            JSONObject jsonObject = (JSONObject) jsonArray.get(0);
            JSONArray urlJSON = JSON.parseArray(jsonObject.get("data").toString());
            downloadUrl = ((JSONObject) urlJSON.get(0)).get("url").toString();
        } catch (Exception e) {
            return "fail";
        }
        return downloadUrl;
    }

    /**
     * 通過url下載文件到本地
    **/
    public static String  downloadFileByUrls(String urlStr,String fileName,String savePath){
        try {
            URL url = new URL(urlStr);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            //設置超時間為3秒
            conn.setConnectTimeout(3 * 1000);
            //防止屏蔽程序抓取而返回403錯誤
            conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");
            //得到輸入流
            InputStream inputStream = conn.getInputStream();
            //獲取自己數組
            byte[] getData = readInputStream(inputStream);
            //文件保存位置
            File saveDir = new File(savePath);
            if (!saveDir.exists()) {
                saveDir.mkdir();
            }
            File file = new File(saveDir + File.separator + fileName);
            FileOutputStream fos = new FileOutputStream(file);
            fos.write(getData);
            if (fos != null) {
                fos.close();
            }
            if (inputStream != null) {
                inputStream.close();
            }
            return "ok";
        }catch (Exception e){
            e.printStackTrace();
            return "fail";
        }
    }


    /**
     * 從輸入流中獲取字節數組
     */
    public static  byte[] readInputStream(InputStream inputStream) throws IOException {
        byte[] buffer = new byte[1024];
        int len = 0;
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        while((len = inputStream.read(buffer)) != -1) {
            bos.write(buffer, 0, len);
        }
        bos.close();
        return bos.toByteArray();
    }
}

2.數據庫實體,及文本讀取內容處理

chat_message表

SET FOREIGN_KEY_CHECKS=0;
-- ----------------------------
-- Table structure for chat_message
-- ----------------------------
DROP TABLE IF EXISTS `chat_message`;
CREATE TABLE `chat_message` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `msg_id` varchar(25) DEFAULT NULL,
  `timestamp` datetime DEFAULT NULL,
  `direction` varchar(50) DEFAULT NULL,
  `to_user` varchar(50) DEFAULT NULL,
  `from_user` varchar(50) DEFAULT NULL,
  `msg` varchar(255) DEFAULT NULL,
  `type` varchar(20) DEFAULT NULL,
  `url` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8;

文本處理

/**
     * 處理環信返回的內容,寫入實體
     *
     * @param content
     */
    @Override
    public void handleContent(String content) {
        JSONArray jsonArray = JSON.parseArray("[" + content + "]");
        List chatMessageList = new ArrayList();
        for(int i = 0; i < jsonArray.size(); i++){
            ChatMessage chatMessage = new ChatMessage();
            JSONObject jsonObject = (JSONObject) jsonArray.get(i);
            JSONArray bodyJsons = (JSONArray)((JSONObject) jsonObject.get("payload")).get("bodies");
            for(int j = 0; j < bodyJsons.size(); j ++) {
                JSONObject bodyJson = (JSONObject) bodyJsons.get(j);
                chatMessage.setMsgId(jsonObject.getString("msg_id"));
                chatMessage.setTimestamp(new Date(Long.parseLong(jsonObject.getString("timestamp"))));
                chatMessage.setDirection(jsonObject.getString("direction"));
                chatMessage.setToUser(jsonObject.getString("to"));
                chatMessage.setFromUser(jsonObject.getString("from"));
                chatMessage.setMsg(bodyJson.getString("msg"));
                chatMessage.setType(bodyJson.getString("type"));
                chatMessage.setUrl(bodyJson.getString("url"));
                chatMessageList.add(chatMessage);
            }
        }
        //批量插入到數據庫
        getMapper().insertBatch(chatMessageList);
    }

用到了mybatis批量插入數據庫,貼一下chatMessageMapper的這一段

    insert into chat_message (msg_id, timestamp, direction, to_user, from_user, msg, type, url)
    values
    
      (#{item.msgId,jdbcType=VARCHAR}, #{item.timestamp,jdbcType=TIMESTAMP},
      #{item.direction,jdbcType=VARCHAR},#{item.toUser,jdbcType=VARCHAR},#{item.fromUser,jdbcType=VARCHAR},
      #{item.msg,jdbcType=VARCHAR},#{item.type,jdbcType=VARCHAR},#{item.url,jdbcType=VARCHAR})
    
  

3.設置一個定時器定時執行service,還可以根據實際項目需求設置定時清理從環信下載的壓縮包文件.

定時器

/**
 * 定時器實現
 *
 * @author Ray
 * @date 2018/1/27 10:35
 */
@Component
public class Timer implements ITimer{

    private static Logger log = LoggerFactory.getLogger(Timer.class);

    @Autowired
    IOrderService orderService;
    @Autowired
    ICouponService couponService;
    @Autowired
    IEaseMobService easeMobService;

    /*
1 Seconds (0-59)
2 Minutes (0-59)
3 Hours (0-23)
4 Day of month (1-31)
5 Month (1-12 or JAN-DEC)
6 Day of week (1-7 or SUN-SAT)
7 Year (1970-2099)
    取值：可以是單個值，如6；
    也可以是個范圍，如9-12；
    也可以是個列表，如9,11,13
    也可以是任意取值，使用*
*/
    @Scheduled(cron = "0 0 12 * * ?")
    public void everyDay() {
        log.info("每日定時器執行");
        //1.檢查訂單自動收貨
        orderService.checkReceiveConfirm();
        log.info("檢查訂單自動收貨");
        //2.失效用戶優惠券
        couponService.updateCouponUseStatusOnTime();
        log.info("失效用戶優惠券");
        //3.保存前一天聊天記錄
        easeMobService.saveChatMessages();
        log.info("保存前一天聊天記錄");
    }
}