본문 바로가기

Lang

MS Azure Blob Storage 에 json 데이터 업로드

DB 데이터를 추출해서 MS Azure Blob Storage 에 json 파일로 저장하는 방법.

from sqlalchemy import create_engine
import pandas as pd
from azure.storage.blob import BlobClient

url = 'YOUR_DB_URL_STR'
engine = create_engine(url)

str_qry = 'SELECT ... FROM ...'
df = pd.read_sql_query(str_qry, engine)
json_data = df.to_json(orient='records')

try:
    conn_str = <azure.blob.conn_str>
    container_name = 'YOUR_CONTAINER_NAME'
    blob_name = 'folder1/data01.json'
    blob_client = BlobClient.from_connection_string(conn_str=conn_str, container_name=container_name, blob_name=blob_name)

    blob_client.upload_blob(json_data, blob_type="BlockBlob")
except Exception as e:
    print(e)

MS 의 Blob Storage API V2.1 이하는 아래처럼 'create_blob_from_text' 를 이용해서 처리하는 것 같은데 BlockBlobService 패키지가 없어져서 BlobClient 의 upload_blob 을 이용해서 테스트해 본 소스임.

BlockBlobService.create_blob_from_text(container_name, blob_name,text, encoding)

blobService = BlockBlobService(account_name=..., account_key=...)
blob_client.create_blob_from_text(container_name, blob_name, json_data)