ã¢ããªãã£ã¯ã¹ããã³ããžãã¹ã€ã³ããªãžã§ã³ã¹ã®ããã®ããŒã¿ãŠã§ã¢ããŠãžã³ã°ãœãªã¥ãŒã·ã§ã³ãSnowflakeãBigQueryãRedshift ã䜿çšããŠãææ°ã®ããŒã¿ãŠã§ã¢ããŠã¹ãèšèšããã³å®è£ ããŸãã
å§ãã
ææ°ã®ããŒã¿ãŠã§ã¢ããŠã¹ã¯ãããŒã¿é§ååæææ±ºå®ã®åºç€ã§ããè€æ°ã®ãœãŒã¹ããã®ããŒã¿ãçµ±åããé«éãªåæã¯ãšãªãå¯èœã«ããè²»çšå¯Ÿå¹æã®é«ãã¹ã±ãŒãªã³ã°ãå®çŸãããŠã§ã¢ããŠã¹ã¢ãŒããã¯ãã£ãèšèšããŸããåœç€Ÿã®ãœãªã¥ãŒã·ã§ã³ã¯ãã¢ãã³ãªããŒã¿ã¹ã¿ã㯠â ELTãã€ãã©ã€ã³ãdbt倿ãããã³ã¬ããã³ã¹ãããã»ãã³ãã£ãã¯ã¬ã€ã€ãŒã䜿çšããŸãã
ã客æ§ã®ã¯ã©ãŠããšã³ã·ã¹ãã ã«å¿ããŠãSnowflakeãBigQueryãRedshiftãåºç€ãšããŠæ§ç¯ããŸãããã€ãã©ã€ã³ãªãŒã±ã¹ãã¬ãŒã·ã§ã³ã¯AirflowãŸãã¯Dagsterã倿ã¯dbtãã€ã³ãžã§ã¹ãã¯FivetranãAirbyteããŸãã¯ã«ã¹ã¿ã ã³ãã¯ã¿ã䜿çšããŸããã¬ããã³ã¹ãšå質ã¯Great ExpectationsãšããŒã¿ã«ã¿ãã°ã§è¡ããŸãã
ã¢ããªãã£ã¯ã¹ãã¬ããŒãäœæãããŒã¿ãµã€ãšã³ã¹ã®ããã«ãè€æ°ã®ãœãŒã¹ããããŒã¿ãçµ±åããå¿ èŠãããäŒæ¥ãæåã®ããŒã¿ãŠã§ã¢ããŠã¹ãæ§ç¯ããå Žåã§ããã¬ã¬ã·ãŒã·ã¹ãã ãã¢ããã€ãºããå Žåã§ããã客æ§ã®ããŒã¿ãšããŒã ã«åãããŠã¹ã±ãŒãªã³ã°ãããœãªã¥ãŒã·ã§ã³ãèšèšããŸãã
ããŒã¿ãœãŒã¹ãæ£åžããåæèŠä»¶ãçè§£ããçŸåšã®ããŒã¿ãããŒããããã³ã°ããŸãã
ãŠã§ã¢ããŠã¹ã¹ããŒãããã€ãã©ã€ã³ã¢ãŒããã¯ãã£ã倿ã¬ã€ã€ãŒãã¬ããã³ã¹ã¢ãã«ãèšèšããŸãã
ã€ã³ãžã§ã¹ããã€ãã©ã€ã³ãæ§ç¯ããdbt倿ãå®è£ ããåæããŒããäœæããŸãã
BIããŒã«ãæ¥ç¶ããããã·ã¥ããŒããæ§ç¯ããããžãã¹ãŠãŒã¶ãŒåãã®ã»ã«ããµãŒãã¹ã¢ããªãã£ã¯ã¹ãèšå®ããŸãã
ããŒã¿å質ãã§ãã¯ãã¢ã¯ã»ã¹å¶åŸ¡ãã³ã¹ãæé©åãããã¥ã¡ã³ãåãå®è£ ããŸãã
ããŒã¿ãçµ±åããããŒã¿é§ååã®æææ±ºå®ã匷åããããŒã¿ãŠã§ã¢ããŠã¹ãæ§ç¯ããŸãããã
SnowflakeãAmazon RedshiftãGoogle BigQueryãAzure Synapse ãªã©ã®ããŒã¿ãŠã§ã¢ããŠã¹ãæ§ç¯ããã»ããApache Hive ã ClickHouse ãšãã£ããªãŒãã³ãœãŒã¹ãœãªã¥ãŒã·ã§ã³ã掻çšããã客æ§ã®åæã¯ãŒã¯ããŒããšäºç®ã«æé©ãªãã©ãããã©ãŒã ãéžæããŸãã
MicrocosmWorks ã¯ãããŒã¿ãŠã§ã¢ããŠãžã³ã°ã®ã³ã³ãµã«ãã£ã³ã°ãšéçºã1æéããã25ãã«ãã50ãã«ã§æäŸããŠããŸãããããžã§ã¯ãè²»çšã¯ãããŒã¿ãœãŒã¹ã®æ°ã倿ã®è€éããã¬ããŒãèŠä»¶ã«ãã£ãŠç°ãªããŸãã
ã¯ããã客æ§ã®ããžãã¹ãã¡ã€ã³ã«åãããŠãKimball ãŸã㯠Inmon ã®ææ³ãçšããŠã¹ã¿ãŒåã¹ããŒããšã¹ããŒãã¬ãŒã¯åã¹ããŒããèšèšããã客æ§ã®åæããŒã ã BI ããŒã ã®ããã«ãå¹ççãªã¯ãšãªããã©ãŒãã³ã¹ãšçŽæçãªããŒã¿æ§é ãä¿èšŒããŸãã
ã客æ§ã®å±¥æŽè¿œè·¡ããŒãºã«å¿ããŠãCDC ããŒã¹ã®å¢åããŒããããŒãž/ã¢ãããµãŒããã¿ãŒã³ãSCD Type 1/2 æŠç¥ãå®è£ ããŸããåœç€Ÿã® ELT ãã€ãã©ã€ã³ã¯ãdbtãAirflowãFivetran ãªã©ã®ããŒã«ã䜿çšããŠä¿¡é Œæ§ã®é«ããªãŒã±ã¹ãã¬ãŒã·ã§ã³ãå®çŸããŸãã
ãã¡ããã§ããã客æ§ã®ãŠã§ã¢ããŠã¹å©çšãã¿ãŒã³ãç£æ»ããã¯ãšãªæ§é ãæé©åãããããªã¢ã©ã€ãºããã¥ãŒãå®è£ ããèªåäžæåæ¢ããªã·ãŒãèšå®ããã³ã³ãã¥ãŒãã£ã³ã°ãªãœãŒã¹ã驿£åããããšã§ãæã ã®ããŒã¿ãŠã§ã¢ããŠã¹ã³ã¹ãã30ã60%åæžããŸãã