Total Pageviews

Showing posts with label Python. Show all posts
Showing posts with label Python. Show all posts

2018/04/06

[Python] namedtuple

Assume I create a Dog class as bellows:
class Dog():
    def __init__(self, size, name):
        self.size = size
        self.name = name

If I would like to set values into Dog class, the code snippet looks like:
pug = Dog('小型犬', '巴哥犬')
shiba = Dog('中型犬', '柴犬')
print('size = {}, name = {}'.format(pug.size, pug.name))
print('size = {}, name = {}'.format(shiba.size, shiba.name))


We can use namedtuple instead of Class.The code snippet are the following:
from collections import namedtuple
# namedtuple instances are just as memory efficient as regular tuples 
# because they do not have per-instance dictionaries. 
# Each kind of namedtuple is represented by its own class, 
# created by using the namedtuple() factory function. 
# The arguments are the name of the new class and a string containing the names of the elements.
Dog = namedtuple('Dog', 'size name')
corgi = Dog('小型犬', '柯基')
husky = Dog('中型犬', '哈士奇')
print('size = {}, name = {}'.format(corgi.size, corgi.name))
print('size = {}, name = {}'.format(husky.size, husky.name))

2018/04/05

[Python] How to parse JSON string?

Problem
I plan to connect to a url, http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023, to get data with JSON format. How to parse JSON string vi Python, and get specific data?


How-to
Here has sample code:
import urllib.request, json
from test.json.row import row

import os
os.environ['http_proxy'] = 'http://proxy.cht.com.tw:8080'

with urllib.request.urlopen("http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023") as url:
    # return with dict data type
    data = json.loads(url.read().decode())
    dataList = []
    count = 1
    # 取得 result / records 下的資料
    for value in data['result']['records']:
        # Skip the first record
        if(count > 1):
            no = value['No']              # 編號
            address = value['Address']    # 地點位置
            deptNm = value['DeptNm']      # 管轄警察局
            branchNm = value['BranchNm']  # 分局
            dataList.append(row(no, address, deptNm, branchNm)) 
            print('no={}, address={}, deptNm={}, branchNm={}'.format(no, address, deptNm, branchNm))
        count += count 


Reference
[1] https://data.gov.tw/dataset/6247



2018/03/10

[Python] How to open a url with urllib via proxy ?

Problem
When I try to open a URL via urllib, but I get this error message:
  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\http\client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 722, in create_connection
    raise err
  File "C:\Users\albert\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 713, in create_connection
    sock.connect(sa)
TimeoutError: [WinError 10060] 連線嘗試失敗因為連線對象有一段時間並未正確回應或是連線建立失敗因為連線的主機無法回應

Here has the code snippet:
import urllib.request, json
from test.json.row import row

with urllib.request.urlopen("http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023") as url:
    data = json.loads(url.read().decode())
    print(data)
    dataList = []
    count = 1
    for value in data['result']['records']:
        if(count > 1):
            no = value['No']              # 編號
            address = value['Address']    # 地點位置
            deptNm = value['DeptNm']      # 管轄警察局
            branchNm = value['BranchNm']  # 分局
            dataList.append(row(no, address, deptNm, branchNm)) 
            print('no={}, address={}, deptNm={}, branchNm={}'.format(no, address, deptNm, branchNm))
        count += count 


How-to
This error result from failing to connect to this URL. I need to configure proxy in my network environment as I connect. Hence, the updated code will look like:
import urllib.request, json
from test.json.row import row

import os
os.environ['http_proxy'] = 'http://proxy.cht.com.tw:8080'

with urllib.request.urlopen("http://od.moi.gov.tw/api/v1/rest/datastore/A01010000C-000628-023") as url:
    data = json.loads(url.read().decode())
    print(data)
    dataList = []
    count = 1
    for value in data['result']['records']:
        if(count > 1):
            no = value['No']              # 編號
            address = value['Address']    # 地點位置
            deptNm = value['DeptNm']      # 管轄警察局
            branchNm = value['BranchNm']  # 分局
            dataList.append(row(no, address, deptNm, branchNm)) 
            print('no={}, address={}, deptNm={}, branchNm={}'.format(no, address, deptNm, branchNm))
        count += count 

2018/03/09

[Python] UnicodeDecodeError: 'cp950' codec can't decode byte 0x99 in position 241: illegal multibyte sequence

Problem
When I try to read a file via Python, I get an error message as following:
1
2
3
4
Traceback (most recent call last):
  File "D:\work\myworkspace\PyTest\src\test\getdata.py", line 11, in <module>
    json = json_file.read()
UnicodeDecodeError: 'cp950' codec can't decode byte 0x99 in position 241: illegal multibyte sequence

Here has the code snippet:
json_file = open("D:/test/test.json", "r")
json = json_file.read()
print(json)        
json_file.close()


How-To
Remember add encoding when opening file, here has the updated code snippet:
json_file = open("D:/test/test.json", "r", encoding='utf8')
json = json_file.read()
print(json)        
json_file.close()


2016/04/05

[Python] How to do sorting in Python

Assume I have a Player Class as bellows:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
'''
Created on 2016/2/3

@author: albert
'''
class Player:
    def __init__(self, team, name, position):
        self.Team = team
        self.Name = name
        self.Position = position
    
    def ToString(self):
        return 'team:' + self.Team + ', name:' + self.Name + ', position:' + self.Position


Here has the code snippet which include add players into list, print players before sort, print players after sorted by player's position and name:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
from com.cht.Player import *
from operator import attrgetter

''' defined a list of Players'''
players = [
           Player('Lamigo', '陳 金 鋒', '指定代打'),
           Player('Lamigo', '林 柏 佑', '投手'),
           Player('中信兄弟', '彭 政 閔', '一壘手'),
           Player('統一7-ELEVEn', '高 志 綱', '捕手'),
           Player('義大犀牛', '林 晨 樺', '投手'),
           Player('統一7-ELEVEn', '陳 鏞 基', '游擊手'),
           Player('Lamigo', '王 柏 融', '左外野手'),
           Player('義大犀牛', '胡 金 龍', '指定打擊'),
           Player('統一7-ELEVEn', '王 鏡 銘', '投手')
           ]

''' print the list of players before sorted'''
print('\n [before sorted] ')
for player in players:
    print(player.ToString())

''' sort players and position and name '''
sortedPlayers = sorted(players, key=attrgetter('Position', 'Name'), reverse=False)
print('\n [after sorted] ')

''' print the list of players after sorted'''
for player in sortedPlayers:
    print(player.ToString())

The log will print as following:
 [before sorted] 
team:Lamigo, name:陳 金 鋒, position:指定代打
team:Lamigo, name:林 柏 佑, position:投手
team:中信兄弟, name:彭 政 閔, position:一壘手
team:統一7-ELEVEn, name:高 志 綱, position:捕手
team:義大犀牛, name:林 晨 樺, position:投手
team:統一7-ELEVEn, name:陳 鏞 基, position:游擊手
team:Lamigo, name:王 柏 融, position:左外野手
team:義大犀牛, name:胡 金 龍, position:指定打擊
team:統一7-ELEVEn, name:王 鏡 銘, position:投手

 [after sorted] 
team:中信兄弟, name:彭 政 閔, position:一壘手
team:Lamigo, name:王 柏 融, position:左外野手
team:義大犀牛, name:林 晨 樺, position:投手
team:Lamigo, name:林 柏 佑, position:投手
team:統一7-ELEVEn, name:王 鏡 銘, position:投手
team:Lamigo, name:陳 金 鋒, position:指定代打
team:義大犀牛, name:胡 金 龍, position:指定打擊
team:統一7-ELEVEn, name:高 志 綱, position:捕手
team:統一7-ELEVEn, name:陳 鏞 基, position:游擊手


Assume I have a number array, string array, datetime array, I would like to sort by ascendant and descendant. 

Here is the code snippets for utility class:
1
2
3
4
5
6
7
''' reverse=False means sort by ascendant '''        
def sortArrayByAsc(dataArray):
    return sorted(dataArray, reverse=False)

''' reverse=True means sort by descendant '''
def sortArrayByDesc(dataArray):
    return sorted(dataArray, reverse=True)


Test the number array:

1
2
3
4
5
6
7
numberArray = [6, 3, 1, 2, 4, 5]
print('\n Original integer array:')
print(numberArray)
print('Sort integer array by asc:' )
print(utils.sortArrayByAsc(numberArray))
print('Sort integer array by desc:' )
print(utils.sortArrayByDesc(numberArray)) 

Test result:
 Original integer array:
[6, 3, 1, 2, 4, 5]
Sort integer array by asc:
[1, 2, 3, 4, 5, 6]
Sort integer array by desc:
[6, 5, 4, 3, 2, 1]


Test the string array:
nameArray = ['Albert', 'Mandy', 'Fiona', 'Ben', 'Jules']
print('\n Original string array:')
print(nameArray)
print('Sort string array by asc:' )
print(utils.sortArrayByAsc(nameArray))
print('Sort string array by desc:' )
print(utils.sortArrayByDesc(nameArray))

Test result:
 Original string array:
['Albert', 'Mandy', 'Fiona', 'Ben', 'Jules']
Sort string array by asc:
['Albert', 'Ben', 'Fiona', 'Jules', 'Mandy']
Sort string array by desc:
['Mandy', 'Jules', 'Fiona', 'Ben', 'Albert']


Test the datetime array:
import datetime
datetime1 = datetime.datetime.strptime('2015-11-11 10:10:10', '%Y-%m-%d %H:%M:%S')
datetime2 = datetime.datetime.strptime('2014-01-11 12:00:00', '%Y-%m-%d %H:%M:%S')
datetime3 = datetime.datetime.strptime('2016-02-17 11:22:00', '%Y-%m-%d %H:%M:%S')
datetimes = [datetime1, datetime2, datetime3]
print('\n Original datetime array:')
print(datetimes)
print('Sort datetime array by asc:' )
print(utils.sortArrayByAsc(datetimes))
print('Sort datetime array by desc:' )
print(utils.sortArrayByDesc(datetimes))

Test result:
 Original datetime array:
[datetime.datetime(2015, 11, 11, 10, 10, 10), datetime.datetime(2014, 1, 11, 12, 0), datetime.datetime(2016, 2, 17, 11, 22)]
Sort datetime array by asc:
[datetime.datetime(2014, 1, 11, 12, 0), datetime.datetime(2015, 11, 11, 10, 10, 10), datetime.datetime(2016, 2, 17, 11, 22)]
Sort datetime array by desc:
[datetime.datetime(2016, 2, 17, 11, 22), datetime.datetime(2015, 11, 11, 10, 10, 10), datetime.datetime(2014, 1, 11, 12, 0)]


Test the unique list:
names = ['Albert', 'Mandy', 'Mandy', 'Fiona', 'Ben', 'Jules']
print('\n Original name array:')
print(names)
print('Unique name array:')
print(utils.removeDuplicateInList(names))

Test result:
 Original name array:
['Albert', 'Mandy', 'Mandy', 'Fiona', 'Ben', 'Jules']
Unique name array:
['Fiona', 'Ben', 'Mandy', 'Jules', 'Albert']




2016/04/04

[Python] Using Pickle to read/write file

Using pickle is straightforward: import the required module, then use dump() to save your data and, some time later, load() to restore it. 

The only requirement when working with pickled files is that they have to be opened in binary access mode

Here is the code snippet:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
''' utilize pickle to write data into file '''
def writeFileWithPickle(path, file, data):
    pickle.dump(data, open(path + '/' + file, 'wb'))

''' utilize picke to read data from file '''
def readFileWithPickle(path, file):
    return pickle.load(open(path + '/' + file, 'rb'))

''' utilize pickle to append data to existing file '''
def appendDataWithPickle(path, file, data):
    try:
        with open(path + '/' + file, 'rb') as fileObject:
            list = pickle.load(fileObject)
            for item in data:
                list.append(item)
            writeFileWithPickle(path, file, list)
    except pickle.PickleError as err:
        print('Fail to append data. Error:' + str(err))
    finally:
        if 'fileObject' in locals():
            fileObject.close()


Reference
[1] http://pythontips.com/2013/08/02/what-is-pickle-in-python/


2016/04/03

[Python] Install Python IDE for Eclipse


PyDev is a plugin that enables Eclipse to be used as a Python IDE (supporting also Jython and IronPython).

It uses advanced type inference techniques to provide features such code completion and code analysis, while still providing many others such as a debugger, interactive console, refactoring, etc.
- See more at: https://marketplace.eclipse.org/content/pydev-python-ide-eclipse#sthash.0052eVTc.dpuf

Here is the installation steps:
1. Open Eclipse
2. Help => Install new software


3. Click Add button


4. Fill in name and location, then click OK button


5. Check PyDev and click Next button


6. Click Next


7. Choose I accept option and click Next button




After install PyDev plugin, you need to restart your Eclipse

8. Configure Intercepter in preference


9. Choose the interpreter you have installed in your computer



Create a PyDev Module
1. create a new PyDevProject


2. Fill in project name, and choose Grammar version & intercepter


3. Create a utility module which name myUtils


The python code is as bellows:
'''
Created on 2016年1月24日

@author: albert
'''
def printHelloMessage(name):
    print('Hello! ' + name)
    
def addTwoNumbers(num1, num2):    
    return num1 + num2

4. Create another test client module to call myUtils





The python code is as following:
'''
Created on 2016年1月24日

@author: albert
'''
from utils import myUtils

if __name__ == '__main__':
    myUtils.printHelloMessage("Albert")
    
    print(myUtils.addTwoNumbers(10, 20))

Here is the test result:

2016/03/09

[Python] File I/O example

Example 1: Read file and print its content

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import os

""" 列印檔案內容 (print file content) """
def printFileContent(file):
    try:
        if(isFileExists("D:\\python\\txt\\" + file)==True):
            """ r stands for read """
            data = open("D:\\python\\txt\\" + file, "r", encoding="utf-8")
            for each_line in data:
                print(each_line)
        
        else:
            print("查無檔案 : "+file)
    except IOError as err:
        print('IOError! ' + err.args[0])
    finally:
       data.close()

"""" 檢查檔案是否存在 """
def isFileExists(file):
 if os.path.exists(file):
  return True
 else:
  return False

Execution result:
1
2
3
4
5
6
7
>>> import myUtil
>>> myUtil.printFileContent("AJokeADay.txt")
A rancher asked his veterinarian for some free advice. 

I have a horse that walks normally sometimes, and sometimes he limps. What shall I do?

The Vet replied, The next time he walks normally, sell him.


Example 2: append some words at the end of file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import os

""" 在檔案內容末端增加文字 (add words at the end of file content) """
def appendString(file, content):
    try:
        if(isFileExists("D:\\python\\txt\\" + file) == True):
            """ a stands for append """
            data = open("D:\\python\\txt\\" + file, "a", encoding="utf-8")
            data.write("\n" + content)
        else:
            print("查無檔案:"+file)
    except IOError as err:
        print('IOError! ' + err.args[0])
    finally:
       data.close()  


""" 列印檔案內容 (print file content) """
def printFileContent(file):
    try:
        if(isFileExists("D:\\python\\txt\\" + file)==True):
            """ r stands for read """
            data = open("D:\\python\\txt\\" + file, "r", encoding="utf-8")
            for each_line in data:
                print(each_line)
        
        else:
            print("查無檔案 : "+file)
    except IOError as err:
        print('IOError! ' + err.args[0])
    finally:
       data.close()


"""" 檢查檔案是否存在 """
def isFileExists(file):
 if os.path.exists(file):
  return True
 else:
  return False

Execution result:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
>>> import time
>>> addedContent = time.strftime("%Y/%m/%d")+" edited"
>>> myUtil.appendString("AJokeADay.txt", addedContent)
>>> 
>>> myUtil.printFileContent("AJokeADay.txt")
A rancher asked his veterinarian for some free advice. 

I have a horse that walks normally sometimes, and sometimes he limps. What shall I do?

The Vet replied, The next time he walks normally, sell him.

2015/12/23 edited
>>> 

Example 3. Apply with statement to open file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
def printFileContent2(fileName):
    try:
        if(isFileExists("D:\\python\\txt\\" + fileName)==True):
            """ r stands for read """
            with open("D:\\python\\txt\\" + fileName, "r", encoding="utf-8") as data:
                for each_line in data:
                    print(each_line)
        
        else:
            print("查無檔案 : "+fileName)
    except IOError as err:
        print('IOError! ' + str(err))
    finally:
       if 'data' in locals():
        data.close()
    







2016/03/08

[Python] IF...ELIF...ELSE Statements

Example
Assume I have four exam type, each type has its passing scores. 
I will provide a method, as people fill in exam type and scores, it will you tell you pass the exam or not. 
This example will demonstrate how to use if / elif / else in Python program:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
""" 判斷分數是否通過考試 """
def passExan(examType, scores):
 if examType == '1':
  if scores >= 252.79:
   print("恭喜您通過 專業1→營運職2 升等考試")
  else:
   print("很抱歉,您未通過 專業1→營運職2 升等考試")
 elif examType == '2':
  if scores >= 248.65:
   print("恭喜您通過 專業2→專業1 升等考試")
  else:
   print("很抱歉,您未通過 專業2→專業1 升等考試")
 elif examType == '3':
  if scores >= 260.79:
   print("恭喜您通過 專業3→專業2 升等考試")
  else:
   print("很抱歉,您未通過 專業3→專業2 升等考試")
 elif examType == '4':
  if scores >= 254.88:
   print("恭喜您通過 專業4→專業3 升等考試")
  else:
   print("很抱歉,您未通過 專業4→專業3 升等考試")
 else:
  print("查無此考試項目")

Test:
1
2
3
4
5
6
7
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import myUtil
>>> myUtil.passExam("1", 270)
恭喜您通過 專業1→營運職2 升等考試
>>> myUtil.passExam("3", 180)
很抱歉,您未通過 專業3→專業2 升等考試


Reference
[1] http://www.tutorialspoint.com/python/python_if_else.htm