albert's blog

2018/03/11

[Kindle] Cannot Find Word Wise Function in eBook

Problem
I had order an eBook, Thinking in Systems: A Primer, from Amazon.com.
According to its product specification, this eBook support word wise

If this eBook support word wise, you can find word wise in the lower right corner

But the eBook which I bought cannot find Word Wise function in the lower right corner.

I contacted with Amazon customer service. Customer service had confirmed that the eBook is defective, they need to contact the publisher about updating the eBook. They will me once Amazon get any update about the eBook, then I can place the order for the eBook once it is updated.

Problem resolution process is as following:

2018/03/10

[Python] How to open a url with urllib via proxy ?

Problem
When I try to open a URL via urllib, but I get this error message:

  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\http\client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 722, in create_connection
    raise err
  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 713, in create_connection
    sock.connect(sa)
TimeoutError: [WinError 10060] 連線嘗試失敗，因為連線對象有一段時間並未正確回應，或是連線建立失敗，因為連線的主機無法回應。

Here has the code snippet:

import urllib.request, json
from test.json.row import row

with urllib.request.urlopen("http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023") as url:
    data = json.loads(url.read().decode())
    print(data)
    dataList = []
    count = 1
    for value in data['result']['records']:
        if(count > 1):
            no = value['No']              # 編號
            address = value['Address']    # 地點位置
            deptNm = value['DeptNm']      # 管轄警察局
            branchNm = value['BranchNm']  # 分局
            dataList.append(row(no, address, deptNm, branchNm)) 
            print('no={}, address={}, deptNm={}, branchNm={}'.format(no, address, deptNm, branchNm))
        count += count

How-to
This error result from failing to connect to this URL. I need to configure proxy in my network environment as I connect. Hence, the updated code will look like:

import urllib.request, json
from test.json.row import row

import os
os.environ['http_proxy'] = 'http://proxy.cht.com.tw:8080'

with urllib.request.urlopen("http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023") as url:
    data = json.loads(url.read().decode())
    print(data)
    dataList = []
    count = 1
    for value in data['result']['records']:
        if(count > 1):
            no = value['No']              # 編號
            address = value['Address']    # 地點位置
            deptNm = value['DeptNm']      # 管轄警察局
            branchNm = value['BranchNm']  # 分局
            dataList.append(row(no, address, deptNm, branchNm)) 
            print('no={}, address={}, deptNm={}, branchNm={}'.format(no, address, deptNm, branchNm))
        count += count

2018/03/09

[Python] UnicodeDecodeError: 'cp950' codec can't decode byte 0x99 in position 241: illegal multibyte sequence

Problem
When I try to read a file via Python, I get an error message as following:

Traceback (most recent call last):
  File "D:\work\myworkspace\PyTest\src\test\getdata.py", line 11, in <module>
    json = json_file.read()
UnicodeDecodeError: 'cp950' codec can't decode byte 0x99 in position 241: illegal multibyte sequence

Here has the code snippet:

json_file = open("D:/test/test.json", "r")
json = json_file.read()
print(json)        
json_file.close()

How-To
Remember add encoding when opening file, here has the updated code snippet:

json_file = open("D:/test/test.json", "r", encoding='utf8')
json = json_file.read()
print(json)        
json_file.close()

2018/03/08

[Java] Utilize JAXB to Read / Write XML

Scenario

The XML content looks like:

  <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
  <products>
      <product id="P01">
          <description>Sony Alpha 7 II</description>
          <price>1620</price>
          <createdBy>
              <email>albert@gmail.com</email>
              <id>1</id>
              <name>Albert</name>
          </createdBy>
      </product>
      <product id="P02">
          <description>Sony Alpha 7R II</description>
          <price>2585</price>
          <createdBy>
              <email>albert@gmail.com</email>
              <id>1</id>
              <name>Albert</name>
          </createdBy>
      </product>
  </products>

How-To
Create three POJOs:

  package test.albert.xml;
  
  import java.util.ArrayList;
  import java.util.List;
  
  import javax.xml.bind.annotation.XmlElement;
  import javax.xml.bind.annotation.XmlRootElement;
  
  import lombok.ToString;
  
  @XmlRootElement(name = "products")
  @ToString
  public class Products {
      List<Product> products;
  
      public List<Product> getProducts() {
          return products;
      }
  
      @XmlElement(name = "product")
      public void setProducts(List<Product> products) {
          this.products = products;
      }
  
      public void add(Product product) {
          if (this.products == null) {
              this.products = new ArrayList<Product>();
          }
          this.products.add(product);
      }
      
  }

  package test.albert.xml;
  
  import java.math.BigDecimal;
  
  import javax.xml.bind.annotation.XmlAttribute;
  import javax.xml.bind.annotation.XmlElement;
  import javax.xml.bind.annotation.XmlRootElement;
  
  import lombok.AllArgsConstructor;
  import lombok.NoArgsConstructor;
  import lombok.ToString;
  
  /**
   * @XmlRootElement: This annotation is used at the top level class to indicate
   *                  the root element in the XML document. The name attribute in
   *                  the annotation is optional. If not specified, the class name
   *                  is used as the root XML element in the document.
   * @XmlAttribute: This annotation is used to indicate the attribute of the root
   *                element.
   * @XmlElement: This annotation is used on the properties of the class that will
   *              be the sub-elements of the root element.
   *
   */
  @AllArgsConstructor
  @NoArgsConstructor
  @ToString
  @XmlRootElement(name = "product")
  public class Product {
  
      @XmlAttribute(name = "id")
      private String productId;
  
      @XmlElement(name = "description")
      private String description;
  
      @XmlElement(name = "price")
      private BigDecimal price;
  
      @XmlElement(name = "createdBy")
      private User createdBy;
  
  }

  package test.albert.xml;
  
  import lombok.AllArgsConstructor;
  import lombok.Data;
  import lombok.NoArgsConstructor;
  import lombok.ToString;
  
  @ToString
  @NoArgsConstructor
  @AllArgsConstructor
  @Data
  public class User {
      private Long id;
      private String name;
      private String email;
  }

Create a test case to implement XML marshalling and unmarshalling:

  package test.albert.xml;
  
  import java.io.File;
  import java.math.BigDecimal;
  
  import javax.xml.bind.JAXBContext;
  import javax.xml.bind.JAXBException;
  import javax.xml.bind.Marshaller;
  import javax.xml.bind.Unmarshaller;
  
  import org.junit.After;
  import org.junit.Before;
  import org.junit.Test;
  
  import lombok.extern.slf4j.Slf4j;
  
  @Slf4j
  public class JAXBTest {
  
      private Product sonayA7ii;
      private Product sonayA7Rii;
      private String xmlFile;
  
      @Before
      public void setUp() {
          User albert = new User(1L, "Albert", "albert@gmail.com");
          sonayA7ii = new Product("P01", "Sony Alpha 7 II", new BigDecimal(1620), albert);
          sonayA7Rii = new Product("P02", "Sony Alpha 7R II", new BigDecimal(2585), albert);
          xmlFile = "F:" + File.separator + "product.xml";
      }
  
      @Test
      public void testObjectToXml() throws JAXBException {
  
          Products products = new Products();
          products.add(sonayA7ii);
          products.add(sonayA7Rii);
  
          JAXBContext jaxbContext = JAXBContext.newInstance(Products.class);
          Marshaller marshaller = jaxbContext.createMarshaller();
          marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
  
          marshaller.marshal(products, new File(xmlFile));
          marshaller.marshal(products, System.out);
      }
  
      @Test
      public void testXmlToObject() throws JAXBException {
          File file = new File(xmlFile);
          JAXBContext jaxbContext = JAXBContext.newInstance(Products.class);
          Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
          
          Products products = (Products) unmarshaller.unmarshal(file);
          log.debug(products.toString());
      }
  
      @After
      public void tearDown() {
          sonayA7ii = null;
          sonayA7Rii = null;
          xmlFile = "";
      }
  
  }

2018/03/07

[commons-validator] numeric-related utilies

Scenario
In real world, we may need some common functions to fulfill our requirement, such as:
1. Validate the number string (including Integer and Double)
2. Convert Integer and Double to String, and apply 1000 sepearator
3. Convert string to Integer and Double
4. Check the number is in specific range

We can use commons-validator to implement this function easily.

Remember to add dependency in your pom.xml

  <dependency>
   <groupId>commons-validator</groupId>
   <artifactId>commons-validator</artifactId>
   <version>1.6</version>
  </dependency>

Sample Code

package albert.practice.validation;

import org.apache.commons.validator.routines.DoubleValidator;
import org.apache.commons.validator.routines.IntegerValidator;

public class NumericUtils {

    /**
     * 檢查輸入的 integer string 是否為數字.
     * 
     * @param number
     */
    public static void isValidInteger(String number) {
       if (IntegerValidator.getInstance().validate(number) == null) {
           throw new IllegalArgumentException("您輸入的非數字");
       }
    }

    /**
     * 檢查輸入的 double string 是否為數字.
     * 
     * @param number
    */
    public static void isValidDouble(String number) {
       if (DoubleValidator.getInstance().validate(number) == null) {
           throw new IllegalArgumentException("您輸入的非數字");
       }
    }

    /**
     * 將 integer 加上 1000 separator.
     * 
     * @param number
     * @return
     */
    public static String formatInteger(Integer number) {
        return IntegerValidator.getInstance().format(number, "#,##0");
    }

    /**
     * 將 Double 加上 1000 separator.
     * 
     * @param number
     * @return
     */
    public static String formatDouble(Double number) {
        return DoubleValidator.getInstance().format(number, "#,##0.00");
    }

    /**
     * 將integer string 轉成 Integer.
     * 
     * @param numberStr
     * @return
     */
    public static Integer formatIntegerString(String numberStr) {
        return IntegerValidator.getInstance().validate(numberStr, "#,##0");
    }

    /**
     * 將double string 轉成 Double.
     * 
     * @param numberStr
     * @return
     */
    public static Double formatDoubleString(String numberStr) {
        return DoubleValidator.getInstance().validate(numberStr, "#,##0.00");
    }

    /**
     * 檢查 integer 是否介於指定區間.
     * 
     * @param number
     * @param min
     * @param max
     */
    public static void isIntegerInValidRange(Integer number, int min, int max) {
        if (!IntegerValidator.getInstance().isInRange(number, min, max)) {
            throw new IllegalArgumentException("您輸入的數字必須介於 " + min + " ~ " + max + " 之間");
        }
    }

    /**
     * 檢查 double 是否介於指定區間.
     * 
     * @param number
     * @param min
     * @param max
     */
    public static void isDoubleInValidRange(Double number, double min, double max) {
        if (!DoubleValidator.getInstance().isInRange(number, min, max)) {
            throw new IllegalArgumentException("您輸入的數字必須介於 " + min + " ~ " + max + " 之間");
        }
    }

}

Test Code

package albert.practice.validation;

import static org.junit.Assert.assertEquals;

import org.junit.Before;
import org.junit.Test;

public class NumericUtilsTest {

    private String integerString;
    private String invalidIntegerString;

    private String doubleString;
    private String invaliddoubleString;

    @Before
    public void setup() {
       integerString = "12345000";
       invalidIntegerString = "1000a";

       doubleString = "12400.60";
       invaliddoubleString = "12400a.60";
    }

    @Test
    public void testValidInteger() {
       NumericUtils.isValidInteger(integerString);
    }

    @Test(expected = IllegalArgumentException.class)
    public void testInvalidInteger() {
       NumericUtils.isValidInteger(invalidIntegerString);
    }

    @Test
    public void testFormatIntegerToString() {
       String numberString = NumericUtils.formatInteger(12345600);
       assertEquals("12,345,600", numberString);
    }

    @Test
    public void testFormatNumStringToInteger() {
       Integer number = NumericUtils.formatIntegerString("12,345,600");
       assertEquals(new Integer(12345600), number);
    }

    @Test
    public void testIntegerIsInRange() {
       NumericUtils.isIntegerInValidRange(new Integer(22), 20, 30);
    }

    @Test(expected = IllegalArgumentException.class)
    public void testIntegerIsNotInRange() {
       NumericUtils.isIntegerInValidRange(new Integer(35), 20, 30);
    }

    @Test
    public void testValidDouble() {
       NumericUtils.isValidDouble(doubleString);
    }

    @Test(expected = IllegalArgumentException.class)
    public void testInvalidDouble() {
       NumericUtils.isValidDouble(invaliddoubleString);
    }

    @Test
    public void testFormatDouble(){
       String number = NumericUtils.formatDouble(new Double(12300));
       assertEquals("12,300.00", number);
    }
 
    @Test
    public void testFormatStringToDouble() {
       Double number = NumericUtils.formatDoubleString(doubleString);
       assertEquals(new Double(12400.60), number);
    }

    @Test
    public void testDoubleIsInRange() {
       NumericUtils.isDoubleInValidRange(new Double(20.5), 20.0, 30.0);
    }
 
    @Test(expected = IllegalArgumentException.class)
    public void testDoubleIsNotInRange() {
       NumericUtils.isDoubleInValidRange(new Double(30.5), 20.0, 30.0);
    }

}

2018/03/06

[commons-validator] Date-related utilies

Scenario
In real world, we may need some common functions to fulfill our requirement, such as:
1. check date string has expected format
2. convert date string to java.util.Date, and vice versa.
3. validate start and date time

We can use commons-validator to implement this function easily.

Remember to add dependency in your pom.xml

 <dependency>
  <groupId>commons-validator</groupId>
  <artifactId>commons-validator</artifactId>
  <version>1.6</version>
 </dependency>

Sample code:

package albert.practice.validation;

import java.util.Date;

import org.apache.commons.validator.routines.DateValidator;

public class DateUtils {

    /**
     * 檢查 date string 格式是否符合 yyyy/MM/dd 格式.
     * 
     * @param dateString
     */
     public static void validateDate(String dateString) {
         if (!DateValidator.getInstance().isValid(dateString, "yyyy/MM/dd")) {
             throw new IllegalArgumentException("Valid date format is yyyy/MM/dd");
         }
     }

    /**
     * 檢查 date string 格式是否符合 yyyy/MM/dd 格式, 若符合則回傳 java.util.Date
     * 
     * @param dateString
     *            ex. 2017/9/14
     * @return java.util.Date
     */
     public static Date convertToDate(String dateString) {
         validateDate(dateString);
         return DateValidator.getInstance().validate(dateString, "yyyy/MM/dd");
     }

    /**
     * 將 Date 轉成指定格式 (ex. yyyy/MM/dd)的 date string.
     * 
     * @param date
     *            java.util.Date
     * @param pattern
     *            ex. yyyy/MM/dd
     * @return date String
     */
     public static String convertToString(Date date, String pattern) {
         return DateValidator.getInstance().format(date, pattern);
     }

    /**
     * 時間起迄檢查.
     * 
     * @param startDate
     *            開始日期
     * @param endDate
     *            結束日期
     */
     public static void compareDays(Date startDate, Date endDate) {
         int compare = DateValidator.getInstance().compareDates(startDate, endDate, null);
         if (compare > 0) {
             throw new IllegalArgumentException("結束時間不可早於開始時間");
         }
     }
}

Test class:

package albert.practice.validation;

import static org.junit.Assert.assertEquals;

import java.util.Date;

import org.junit.Before;
import org.junit.Test;

public class DateUtilsTest {

    private String validDate = "";
    private String invalidDate = "";
    private String startDate = "";
    private String endDate = "";
    private String invalidEndDate = "";

    @Before
    public void setup() {
        validDate = "2017/09/30";
        invalidDate = "2017/09/31";

        startDate = "2017/9/14";
        endDate = "2017/9/15";
        invalidEndDate = "2017/9/11";
    }

    @Test(expected = IllegalArgumentException.class)
    public void testInvalidDate() {
        DateUtils.validateDate(invalidDate);
    }

    @Test
    public void testValidDate() {
        DateUtils.validateDate(validDate);
    }

    @Test
    public void testDateConversion() {
        Date date = DateUtils.convertToDate(validDate);
        String dateString = DateUtils.convertToString(date, "yyyy/MM/dd");
        assertEquals("2017/09/30", dateString);
    }

    @Test
    public void testValidStartAndEndDate() {
        DateUtils.compareDays(DateUtils.convertToDate(startDate), DateUtils.convertToDate(endDate));
    }

    @Test(expected = IllegalArgumentException.class)
    public void testInvalidStartAndEndDate() {
        DateUtils.compareDays(DateUtils.convertToDate(startDate), DateUtils.convertToDate(invalidEndDate));
    }

}

Total Pageviews