albert's blog

2018/03/13

[jsoup] How to use jsoup selector to retrieve value?

Problem
I try to get "2412 中華電" from this page: https://goodinfo.tw/StockInfo/StockDetail.asp?STOCK_ID=2412

jsoup provide select method to find elements that match the Selector CSS query, with this element as the starting context. Matched elements may include this element, or any of its children. But how to define the CSS selector?

    public static void main(String[] args) throws IOException {
        String cssSelector = "???";
        String url = "https://goodinfo.tw/StockInfo/StockDetail.asp?STOCK_ID=2412";
        Document doc = Jsoup.connect(url).get();
        String name = doc.select(cssSelector).text();
    }

How-To
You can use Google Chrome to help you get the CSS Selector:

Here has the sample code:

    public static void main(String[] args) throws IOException {
        String cssSelector = "body > table:nth-child(3) > tbody > tr > td:nth-child(3) > table > tbody > tr:nth-child(1) > td > table:nth-child(1) > tbody > tr > td > table > tbody > tr:nth-child(1) > td > span:nth-child(1) > a";
        String url = "https://goodinfo.tw/StockInfo/StockDetail.asp?STOCK_ID=2412";
        Document doc = Jsoup.connect(url).get();
        String name = doc.select(cssSelector).text();
    }

2018/03/12

[snakeyaml] How to generate YAML via Java?

Problem
How to generate YAML via Java?
The expected YAML content looks like:

  intents:
  - greet
  - inform
  - others
  - question
  entities:
  - userid
  - is_account_locked
  - pass_company_check

How-To
Add snakeyaml dependency to pom.xml

 <dependency>
  <groupId>org.yaml</groupId>
  <artifactId>snakeyaml</artifactId>
  <version>1.19</version>
 </dependency>

Here has sample code which implement via snakeyaml

  package test.albert.snakeyaml;
  
  import java.util.Arrays;
  import java.util.HashMap;
  import java.util.List;
  import java.util.Map;
  
  import org.yaml.snakeyaml.DumperOptions;
  import org.yaml.snakeyaml.Yaml;
  
  import lombok.AllArgsConstructor;
  import lombok.Data;
  import lombok.extern.slf4j.Slf4j;
  
  @Slf4j
  public class SnakeYamlTest {
  
      public static void main(String[] args) {
          // Set DumperOptions options
          DumperOptions options = new DumperOptions();
          options.setDefaultFlowStyle(DumperOptions.FlowStyle.BLOCK);
          options.setPrettyFlow(true);
  
          // Create Yaml instance with DumperOptions
          Yaml yaml = new Yaml(options);
  
          // Prepare map data for YAML
          Map<String, Object> map = new HashMap<>();
          map.put("intents", Arrays.asList("greet", "inform", "others", "question"));
          map.put("entities", Arrays.asList("userid", "is_account_locked", "pass_company_check"));
  
          // Serialize a Java object into a YAML String
          String output = yaml.dump(map);
          log.info("\n" + output);
      }
  }

2018/03/11

[Kindle] Cannot Find Word Wise Function in eBook

Problem
I had order an eBook, Thinking in Systems: A Primer, from Amazon.com.
According to its product specification, this eBook support word wise

If this eBook support word wise, you can find word wise in the lower right corner

But the eBook which I bought cannot find Word Wise function in the lower right corner.

I contacted with Amazon customer service. Customer service had confirmed that the eBook is defective, they need to contact the publisher about updating the eBook. They will me once Amazon get any update about the eBook, then I can place the order for the eBook once it is updated.

Problem resolution process is as following:

2018/03/10

[Python] How to open a url with urllib via proxy ?

Problem
When I try to open a URL via urllib, but I get this error message:

  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\http\client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 722, in create_connection
    raise err
  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 713, in create_connection
    sock.connect(sa)
TimeoutError: [WinError 10060] 連線嘗試失敗，因為連線對象有一段時間並未正確回應，或是連線建立失敗，因為連線的主機無法回應。

Here has the code snippet:

import urllib.request, json
from test.json.row import row

with urllib.request.urlopen("http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023") as url:
    data = json.loads(url.read().decode())
    print(data)
    dataList = []
    count = 1
    for value in data['result']['records']:
        if(count > 1):
            no = value['No']              # 編號
            address = value['Address']    # 地點位置
            deptNm = value['DeptNm']      # 管轄警察局
            branchNm = value['BranchNm']  # 分局
            dataList.append(row(no, address, deptNm, branchNm)) 
            print('no={}, address={}, deptNm={}, branchNm={}'.format(no, address, deptNm, branchNm))
        count += count

How-to
This error result from failing to connect to this URL. I need to configure proxy in my network environment as I connect. Hence, the updated code will look like:

import urllib.request, json
from test.json.row import row

import os
os.environ['http_proxy'] = 'http://proxy.cht.com.tw:8080'

with urllib.request.urlopen("http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023") as url:
    data = json.loads(url.read().decode())
    print(data)
    dataList = []
    count = 1
    for value in data['result']['records']:
        if(count > 1):
            no = value['No']              # 編號
            address = value['Address']    # 地點位置
            deptNm = value['DeptNm']      # 管轄警察局
            branchNm = value['BranchNm']  # 分局
            dataList.append(row(no, address, deptNm, branchNm)) 
            print('no={}, address={}, deptNm={}, branchNm={}'.format(no, address, deptNm, branchNm))
        count += count

2018/03/09

[Python] UnicodeDecodeError: 'cp950' codec can't decode byte 0x99 in position 241: illegal multibyte sequence

Problem
When I try to read a file via Python, I get an error message as following:

Traceback (most recent call last):
  File "D:\work\myworkspace\PyTest\src\test\getdata.py", line 11, in <module>
    json = json_file.read()
UnicodeDecodeError: 'cp950' codec can't decode byte 0x99 in position 241: illegal multibyte sequence

Here has the code snippet:

json_file = open("D:/test/test.json", "r")
json = json_file.read()
print(json)        
json_file.close()

How-To
Remember add encoding when opening file, here has the updated code snippet:

json_file = open("D:/test/test.json", "r", encoding='utf8')
json = json_file.read()
print(json)        
json_file.close()

2018/03/08

[Java] Utilize JAXB to Read / Write XML

Scenario

The XML content looks like:

  <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
  <products>
      <product id="P01">
          <description>Sony Alpha 7 II</description>
          <price>1620</price>
          <createdBy>
              <email>albert@gmail.com</email>
              <id>1</id>
              <name>Albert</name>
          </createdBy>
      </product>
      <product id="P02">
          <description>Sony Alpha 7R II</description>
          <price>2585</price>
          <createdBy>
              <email>albert@gmail.com</email>
              <id>1</id>
              <name>Albert</name>
          </createdBy>
      </product>
  </products>

How-To
Create three POJOs:

  package test.albert.xml;
  
  import java.util.ArrayList;
  import java.util.List;
  
  import javax.xml.bind.annotation.XmlElement;
  import javax.xml.bind.annotation.XmlRootElement;
  
  import lombok.ToString;
  
  @XmlRootElement(name = "products")
  @ToString
  public class Products {
      List<Product> products;
  
      public List<Product> getProducts() {
          return products;
      }
  
      @XmlElement(name = "product")
      public void setProducts(List<Product> products) {
          this.products = products;
      }
  
      public void add(Product product) {
          if (this.products == null) {
              this.products = new ArrayList<Product>();
          }
          this.products.add(product);
      }
      
  }

  package test.albert.xml;
  
  import java.math.BigDecimal;
  
  import javax.xml.bind.annotation.XmlAttribute;
  import javax.xml.bind.annotation.XmlElement;
  import javax.xml.bind.annotation.XmlRootElement;
  
  import lombok.AllArgsConstructor;
  import lombok.NoArgsConstructor;
  import lombok.ToString;
  
  /**
   * @XmlRootElement: This annotation is used at the top level class to indicate
   *                  the root element in the XML document. The name attribute in
   *                  the annotation is optional. If not specified, the class name
   *                  is used as the root XML element in the document.
   * @XmlAttribute: This annotation is used to indicate the attribute of the root
   *                element.
   * @XmlElement: This annotation is used on the properties of the class that will
   *              be the sub-elements of the root element.
   *
   */
  @AllArgsConstructor
  @NoArgsConstructor
  @ToString
  @XmlRootElement(name = "product")
  public class Product {
  
      @XmlAttribute(name = "id")
      private String productId;
  
      @XmlElement(name = "description")
      private String description;
  
      @XmlElement(name = "price")
      private BigDecimal price;
  
      @XmlElement(name = "createdBy")
      private User createdBy;
  
  }

  package test.albert.xml;
  
  import lombok.AllArgsConstructor;
  import lombok.Data;
  import lombok.NoArgsConstructor;
  import lombok.ToString;
  
  @ToString
  @NoArgsConstructor
  @AllArgsConstructor
  @Data
  public class User {
      private Long id;
      private String name;
      private String email;
  }

Create a test case to implement XML marshalling and unmarshalling:

  package test.albert.xml;
  
  import java.io.File;
  import java.math.BigDecimal;
  
  import javax.xml.bind.JAXBContext;
  import javax.xml.bind.JAXBException;
  import javax.xml.bind.Marshaller;
  import javax.xml.bind.Unmarshaller;
  
  import org.junit.After;
  import org.junit.Before;
  import org.junit.Test;
  
  import lombok.extern.slf4j.Slf4j;
  
  @Slf4j
  public class JAXBTest {
  
      private Product sonayA7ii;
      private Product sonayA7Rii;
      private String xmlFile;
  
      @Before
      public void setUp() {
          User albert = new User(1L, "Albert", "albert@gmail.com");
          sonayA7ii = new Product("P01", "Sony Alpha 7 II", new BigDecimal(1620), albert);
          sonayA7Rii = new Product("P02", "Sony Alpha 7R II", new BigDecimal(2585), albert);
          xmlFile = "F:" + File.separator + "product.xml";
      }
  
      @Test
      public void testObjectToXml() throws JAXBException {
  
          Products products = new Products();
          products.add(sonayA7ii);
          products.add(sonayA7Rii);
  
          JAXBContext jaxbContext = JAXBContext.newInstance(Products.class);
          Marshaller marshaller = jaxbContext.createMarshaller();
          marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
  
          marshaller.marshal(products, new File(xmlFile));
          marshaller.marshal(products, System.out);
      }
  
      @Test
      public void testXmlToObject() throws JAXBException {
          File file = new File(xmlFile);
          JAXBContext jaxbContext = JAXBContext.newInstance(Products.class);
          Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
          
          Products products = (Products) unmarshaller.unmarshal(file);
          log.debug(products.toString());
      }
  
      @After
      public void tearDown() {
          sonayA7ii = null;
          sonayA7Rii = null;
          xmlFile = "";
      }
  
  }

2018/03/07

[commons-validator] numeric-related utilies

Scenario
In real world, we may need some common functions to fulfill our requirement, such as:
1. Validate the number string (including Integer and Double)
2. Convert Integer and Double to String, and apply 1000 sepearator
3. Convert string to Integer and Double
4. Check the number is in specific range

We can use commons-validator to implement this function easily.

Remember to add dependency in your pom.xml

  <dependency>
   <groupId>commons-validator</groupId>
   <artifactId>commons-validator</artifactId>
   <version>1.6</version>
  </dependency>

Sample Code

package albert.practice.validation;

import org.apache.commons.validator.routines.DoubleValidator;
import org.apache.commons.validator.routines.IntegerValidator;

public class NumericUtils {

    /**
     * 檢查輸入的 integer string 是否為數字.
     * 
     * @param number
     */
    public static void isValidInteger(String number) {
       if (IntegerValidator.getInstance().validate(number) == null) {
           throw new IllegalArgumentException("您輸入的非數字");
       }
    }

    /**
     * 檢查輸入的 double string 是否為數字.
     * 
     * @param number
    */
    public static void isValidDouble(String number) {
       if (DoubleValidator.getInstance().validate(number) == null) {
           throw new IllegalArgumentException("您輸入的非數字");
       }
    }

    /**
     * 將 integer 加上 1000 separator.
     * 
     * @param number
     * @return
     */
    public static String formatInteger(Integer number) {
        return IntegerValidator.getInstance().format(number, "#,##0");
    }

    /**
     * 將 Double 加上 1000 separator.
     * 
     * @param number
     * @return
     */
    public static String formatDouble(Double number) {
        return DoubleValidator.getInstance().format(number, "#,##0.00");
    }

    /**
     * 將integer string 轉成 Integer.
     * 
     * @param numberStr
     * @return
     */
    public static Integer formatIntegerString(String numberStr) {
        return IntegerValidator.getInstance().validate(numberStr, "#,##0");
    }

    /**
     * 將double string 轉成 Double.
     * 
     * @param numberStr
     * @return
     */
    public static Double formatDoubleString(String numberStr) {
        return DoubleValidator.getInstance().validate(numberStr, "#,##0.00");
    }

    /**
     * 檢查 integer 是否介於指定區間.
     * 
     * @param number
     * @param min
     * @param max
     */
    public static void isIntegerInValidRange(Integer number, int min, int max) {
        if (!IntegerValidator.getInstance().isInRange(number, min, max)) {
            throw new IllegalArgumentException("您輸入的數字必須介於 " + min + " ~ " + max + " 之間");
        }
    }

    /**
     * 檢查 double 是否介於指定區間.
     * 
     * @param number
     * @param min
     * @param max
     */
    public static void isDoubleInValidRange(Double number, double min, double max) {
        if (!DoubleValidator.getInstance().isInRange(number, min, max)) {
            throw new IllegalArgumentException("您輸入的數字必須介於 " + min + " ~ " + max + " 之間");
        }
    }

}

Test Code

package albert.practice.validation;

import static org.junit.Assert.assertEquals;

import org.junit.Before;
import org.junit.Test;

public class NumericUtilsTest {

    private String integerString;
    private String invalidIntegerString;

    private String doubleString;
    private String invaliddoubleString;

    @Before
    public void setup() {
       integerString = "12345000";
       invalidIntegerString = "1000a";

       doubleString = "12400.60";
       invaliddoubleString = "12400a.60";
    }

    @Test
    public void testValidInteger() {
       NumericUtils.isValidInteger(integerString);
    }

    @Test(expected = IllegalArgumentException.class)
    public void testInvalidInteger() {
       NumericUtils.isValidInteger(invalidIntegerString);
    }

    @Test
    public void testFormatIntegerToString() {
       String numberString = NumericUtils.formatInteger(12345600);
       assertEquals("12,345,600", numberString);
    }

    @Test
    public void testFormatNumStringToInteger() {
       Integer number = NumericUtils.formatIntegerString("12,345,600");
       assertEquals(new Integer(12345600), number);
    }

    @Test
    public void testIntegerIsInRange() {
       NumericUtils.isIntegerInValidRange(new Integer(22), 20, 30);
    }

    @Test(expected = IllegalArgumentException.class)
    public void testIntegerIsNotInRange() {
       NumericUtils.isIntegerInValidRange(new Integer(35), 20, 30);
    }

    @Test
    public void testValidDouble() {
       NumericUtils.isValidDouble(doubleString);
    }

    @Test(expected = IllegalArgumentException.class)
    public void testInvalidDouble() {
       NumericUtils.isValidDouble(invaliddoubleString);
    }

    @Test
    public void testFormatDouble(){
       String number = NumericUtils.formatDouble(new Double(12300));
       assertEquals("12,300.00", number);
    }
 
    @Test
    public void testFormatStringToDouble() {
       Double number = NumericUtils.formatDoubleString(doubleString);
       assertEquals(new Double(12400.60), number);
    }

    @Test
    public void testDoubleIsInRange() {
       NumericUtils.isDoubleInValidRange(new Double(20.5), 20.0, 30.0);
    }
 
    @Test(expected = IllegalArgumentException.class)
    public void testDoubleIsNotInRange() {
       NumericUtils.isDoubleInValidRange(new Double(30.5), 20.0, 30.0);
    }

}

Total Pageviews