Andy Kopciuch's Blog
Saturday, February 19, 2005
OLE DB Providers, CVS & VCRs
What a joy it is to work with software that not only does exactly what you expect it to do, but is also easy to work with. My latest activities have been relating to the convergence of a system for MS-SQL to PostgeSQL.
I am not completely free of the ball and chain of SQL Server, but the closer I get to that bright light of a real RDBMS, the more excited I get. I had never experienced importing / exporting to and from MS-SQL prior to this week. I wasn't worried per se, but it was unfamiliar to me. I am quite familiar with the workings of this now.
I have installed PostgreSQL 8.0.1 on my Windows machine. This happens to install the OLE DB provider as well (nice). So now I have the option of exporting directly to a PG database. It was quite easy to setup the DSN for my local machine and start chugging away. There are a couple of tricks to the DSN. By default bools are represented as char fields, you can simply uncheck this option. You can also set some other configuration options regarding bigints, text datatypes etc.
The ODBC provider is not the problem area. Once again due to the non standard MS way of doing things it causes problems. "uniqueidentifier" columns. You may as well forget about it. You can transform them into text datatypes, but you'll be pooched after that. There is apparently a package kicking around that adds "uniqueidentifier" as a type to postgres. It relies on the libuuid library, which isn't available on windows. Why one would want that datatype, other than for a slow conversion is beyond me. Sequences are available and 10^23142341 better than a global unique binary value.
Another pitfall when exporting is the fact the MS-SQL handles case sensitivity in a non standard way. According to the standards SQL is case insensitive. PG handles fields, and tables inside of double quotes to be case sensitive, but the basic useage converts to all lower case. I believe the spec states to handle all uppercase. I'm not sure about that. Regardless ... this causes problems on the table data movement because a column named someID would have been created as someid. But when the data is moved from SQL it's coming from someID and into someID ... which isn't available. You basically have to go slow, and transform all of your column names to be lowercase for table creation.
The other option is to re-create the schema first, and then just append the data into an existing table. From what I have read on the internet this provides much greater results. You can transform data, and shift columns as needed. This may be the way to go when you are moving more critical data. Since my schemas are rather small, and the system is in develeopment, I just dumped the SQL scripts, and removed all of the MS-SQL crap in them (like square brackets [], and dbo designations, and specifying COLLATE blah blah, Latin1_ blah blah for var chars etc.). I think this helps you realize how lacking in features, and datatypes other DBs are compared to PostgreSQL. Not Boolean, numerous date / time types, Text, and a heap of others.
<time days="6" / >
That was me being funny with XML. Alright ... not so funny. I wrote the first half of this in a draft six days ago. Since I'm leaving Sooke to head back to Calgary tomorrow or the next, I decided to sum up my week here before I check out.
I continued on with my PostgeSQL fun this week. I must say that my research was indeed correct, in that loading your schemas prior to data export yielded a much greater success ratio. It was so much easier to just append some conformed data onto an existing table. You could drop out coplumns as needed, and things seem to go smoothly.
Everyone else seems to be really getting on board with the PG niftyness. It just goes to show you how much people who have worked with bad tools, appreciate the good ones. They'll never go back now. buwhahahahahaha!
We have the developers set up with Apache, PHP & PostgeSQL all running on windows, and the Mac OS X 10.3 server setup and running the same. It took a few different packages, and a server re-install, but we are chuggin along like the old chicago line all the same. This setup is so much cleaner, and nicer to work with than before. IMHO. It also means I could do development in Linux if I wanted, as the tools are all OS and freely available. I did notice that apache released a windows installer for 2.0.53 this last week. When I went it was still unavailable, therefore I still have Apache 1.3.33 on my machine. Not to worry ... it works just fine and danady.
Mik and I spent some considerable time setting up a CVS repository. Every seems to be clicking into how it works, and everything is going smoothly. We even tried out Tortoise CVS for windows ... which is pretty cool BTW. http://www.tortoisecvs.org. I still install the win32 binary from http://cvshome.org myself. I'm an old school command line junkie. :-D I configured CVSSPam which is a set of ruby scripts to send out retty CVS commit log emails. I works quite well, and Chris Lang just asked me to take him off the list, as he got 44 CVS commit emails today. That's what you get for having busy developers I guess. ;-)
As for my time in Victoria this round ... it was wonderful. Beautiful weather and I got a lot of cool things done. Marshall and the girls have had the misfortune recently of having _two_ VCRs and a DVD player that all do not work. That makes it kind of hard to watch the taped episode of 24 from Monday night. I went out and the good soul that I am bought them a Phillips DVD / VCR combo player. Needless to say it is a _HUGE_ hit. They love me. Marshall has almost finished building the new room in the basement. He did a smashing job on it. Owen helped with the Dry Wall, and Johnny is painting today. So it was a group effort. I even mudded a little bit last night. Lets just say I should stick to programming. drywalling really isn;'t my forte. ;-)
Cheers,
See you back in Calgary! ¶ 10:04:00 PM 0 comments
I am not completely free of the ball and chain of SQL Server, but the closer I get to that bright light of a real RDBMS, the more excited I get. I had never experienced importing / exporting to and from MS-SQL prior to this week. I wasn't worried per se, but it was unfamiliar to me. I am quite familiar with the workings of this now.
I have installed PostgreSQL 8.0.1 on my Windows machine. This happens to install the OLE DB provider as well (nice). So now I have the option of exporting directly to a PG database. It was quite easy to setup the DSN for my local machine and start chugging away. There are a couple of tricks to the DSN. By default bools are represented as char fields, you can simply uncheck this option. You can also set some other configuration options regarding bigints, text datatypes etc.
The ODBC provider is not the problem area. Once again due to the non standard MS way of doing things it causes problems. "uniqueidentifier" columns. You may as well forget about it. You can transform them into text datatypes, but you'll be pooched after that. There is apparently a package kicking around that adds "uniqueidentifier" as a type to postgres. It relies on the libuuid library, which isn't available on windows. Why one would want that datatype, other than for a slow conversion is beyond me. Sequences are available and 10^23142341 better than a global unique binary value.
Another pitfall when exporting is the fact the MS-SQL handles case sensitivity in a non standard way. According to the standards SQL is case insensitive. PG handles fields, and tables inside of double quotes to be case sensitive, but the basic useage converts to all lower case. I believe the spec states to handle all uppercase. I'm not sure about that. Regardless ... this causes problems on the table data movement because a column named someID would have been created as someid. But when the data is moved from SQL it's coming from someID and into someID ... which isn't available. You basically have to go slow, and transform all of your column names to be lowercase for table creation.
The other option is to re-create the schema first, and then just append the data into an existing table. From what I have read on the internet this provides much greater results. You can transform data, and shift columns as needed. This may be the way to go when you are moving more critical data. Since my schemas are rather small, and the system is in develeopment, I just dumped the SQL scripts, and removed all of the MS-SQL crap in them (like square brackets [], and dbo designations, and specifying COLLATE blah blah, Latin1_ blah blah for var chars etc.). I think this helps you realize how lacking in features, and datatypes other DBs are compared to PostgreSQL. Not Boolean, numerous date / time types, Text, and a heap of others.
<time days="6" / >
That was me being funny with XML. Alright ... not so funny. I wrote the first half of this in a draft six days ago. Since I'm leaving Sooke to head back to Calgary tomorrow or the next, I decided to sum up my week here before I check out.
I continued on with my PostgeSQL fun this week. I must say that my research was indeed correct, in that loading your schemas prior to data export yielded a much greater success ratio. It was so much easier to just append some conformed data onto an existing table. You could drop out coplumns as needed, and things seem to go smoothly.
Everyone else seems to be really getting on board with the PG niftyness. It just goes to show you how much people who have worked with bad tools, appreciate the good ones. They'll never go back now. buwhahahahahaha!
We have the developers set up with Apache, PHP & PostgeSQL all running on windows, and the Mac OS X 10.3 server setup and running the same. It took a few different packages, and a server re-install, but we are chuggin along like the old chicago line all the same. This setup is so much cleaner, and nicer to work with than before. IMHO. It also means I could do development in Linux if I wanted, as the tools are all OS and freely available. I did notice that apache released a windows installer for 2.0.53 this last week. When I went it was still unavailable, therefore I still have Apache 1.3.33 on my machine. Not to worry ... it works just fine and danady.
Mik and I spent some considerable time setting up a CVS repository. Every seems to be clicking into how it works, and everything is going smoothly. We even tried out Tortoise CVS for windows ... which is pretty cool BTW. http://www.tortoisecvs.org. I still install the win32 binary from http://cvshome.org myself. I'm an old school command line junkie. :-D I configured CVSSPam which is a set of ruby scripts to send out retty CVS commit log emails. I works quite well, and Chris Lang just asked me to take him off the list, as he got 44 CVS commit emails today. That's what you get for having busy developers I guess. ;-)
As for my time in Victoria this round ... it was wonderful. Beautiful weather and I got a lot of cool things done. Marshall and the girls have had the misfortune recently of having _two_ VCRs and a DVD player that all do not work. That makes it kind of hard to watch the taped episode of 24 from Monday night. I went out and the good soul that I am bought them a Phillips DVD / VCR combo player. Needless to say it is a _HUGE_ hit. They love me. Marshall has almost finished building the new room in the basement. He did a smashing job on it. Owen helped with the Dry Wall, and Johnny is painting today. So it was a group effort. I even mudded a little bit last night. Lets just say I should stick to programming. drywalling really isn;'t my forte. ;-)
Cheers,
See you back in Calgary! ¶ 10:04:00 PM 0 comments
Thursday, February 17, 2005
XML, Mac OS X &Neo Citron
I recently found a PEAR module for PHP called XML_Serializer, and XML_Unserialiser. What a fantastic module. These classes utilize PHP native type calls to build and format XML strings for you. It's highly configurable (what to use for indents, line feeds, headings, type hints etc.). The XML is spot on exactly what I expect. The object just takes your data object (simple array, class, array of objects) and spits out XML. It's neat, clean, and simple. It also provides the unserializer in which you pass it XML and will return to you native PHP objects containing data. Pretty slick stuff.
I also did some limited investigation into foxpro web services through SOAP. What an attrocity that is! It looks like a ton of work to set it up, and the XML sent is "whacky". I think we'll just end up building a custom com object that will handle the parsing of simple XML. It's way easier than a SOAP setup. Should do the trick nicely.
This week I've been introduced to Mac OS X 10.3. It's pretty slick I must say. I'll also admit that I've been using windows much to much for my liking, so anything with a built in ls command scores huge points. ;-) We have a working CVS pserver, and a working installation of PostgreSQL. We just need to set up Apache, and PHP properly. 10.3 comes with a stock apache 1.3.33 (I think), so an upgrade to apache 2 is in order. Overall the system is fine to work on. If you have worked on one *NIX system, you can pretty much make your way around any other system. I thought it was based on Debian, turns out to be the Mach Kernel. It's a _HUGE_ step up from windows any which way you dice the turkey gizzards.
Getting the data transformation to work from MS_SQL might be a little more tricky. I've got the ODBC connection working, and some things are moved accross. I should have known it would completely barf all over the MS "uniqueidentifier" datatypes. It attempts to create them as "date" types. hahaha. I think I'll have to re-create the schema first, and then just copy the data over separately. That will be my task tomorrow I believe. Fun Fun Fun.
The OS X packaging community seems rather sporadic to me. Several unique packages are available, that may or may not coincide with the stock apple packages, or whatever other package you have installed. This coming from a guy who a year ago still installed everything from source. I guess I have just become to accustomed to a system with nice packaging layouts (like SuSE). I'm too spoiled and now to compile apache from source, and have one GINORMOUS httpd.conf file just seems wrong. I suppose I could make it work smartly, but I'm also lazy. ;-)
I've been feeling under the weather for a couple of days. Actually I feel like ASS! But my new favorite friend is Neo-Citron with Rye. It's totally wicked! you can't even taste the whiskey in it. I prefer the lemon and rye over the cherry flavoured Neo-Citron. But that's just me.
¶ 3:35:00 PM 0 comments
I also did some limited investigation into foxpro web services through SOAP. What an attrocity that is! It looks like a ton of work to set it up, and the XML sent is "whacky". I think we'll just end up building a custom com object that will handle the parsing of simple XML. It's way easier than a SOAP setup. Should do the trick nicely.
This week I've been introduced to Mac OS X 10.3. It's pretty slick I must say. I'll also admit that I've been using windows much to much for my liking, so anything with a built in ls command scores huge points. ;-) We have a working CVS pserver, and a working installation of PostgreSQL. We just need to set up Apache, and PHP properly. 10.3 comes with a stock apache 1.3.33 (I think), so an upgrade to apache 2 is in order. Overall the system is fine to work on. If you have worked on one *NIX system, you can pretty much make your way around any other system. I thought it was based on Debian, turns out to be the Mach Kernel. It's a _HUGE_ step up from windows any which way you dice the turkey gizzards.
Getting the data transformation to work from MS_SQL might be a little more tricky. I've got the ODBC connection working, and some things are moved accross. I should have known it would completely barf all over the MS "uniqueidentifier" datatypes. It attempts to create them as "date" types. hahaha. I think I'll have to re-create the schema first, and then just copy the data over separately. That will be my task tomorrow I believe. Fun Fun Fun.
The OS X packaging community seems rather sporadic to me. Several unique packages are available, that may or may not coincide with the stock apple packages, or whatever other package you have installed. This coming from a guy who a year ago still installed everything from source. I guess I have just become to accustomed to a system with nice packaging layouts (like SuSE). I'm too spoiled and now to compile apache from source, and have one GINORMOUS httpd.conf file just seems wrong. I suppose I could make it work smartly, but I'm also lazy. ;-)
I've been feeling under the weather for a couple of days. Actually I feel like ASS! But my new favorite friend is Neo-Citron with Rye. It's totally wicked! you can't even taste the whiskey in it. I prefer the lemon and rye over the cherry flavoured Neo-Citron. But that's just me.
¶ 3:35:00 PM 0 comments
Saturday, February 05, 2005
Interfaces, Tech Support & Hair Dressers
It's been ages since I added a new entry here. I've been in Calgary for almost 3 weeks now. It has blown past fairly quickly. I haven't had any creative burst of new and nifty things work wise. That's not to say that I have not done any work, it's just been a consistent level of average work.
I completely re-worked the entire interface for the L&Y toolkit. Chris Lang has been a huge influence on my artisitc web skills since I have been working with him. It's not that I was unaware of user interface techniques, or that I couldn't accomplish what I wanted. Coder's (especially in the OSS world) tend to work on what interests them. Playing with CSS, and picky little user interface issues like changing the font one size smaller, or the font family to something different, and the background shading, and borders, and positioning etc. etc. etc. is not interesting to me. To me it is menial task. My time is better spent doing what I am good at, which is writing code.
I know that all of those things I mentioned are very, very important. Greatly important on a public website. I am a low level coder however, and what interests me is making the web site functional. It's the microsoft vs linux console scenario. I don't care if it looks pretty, the crappy software doesn't do what I need it to. Hey ... over here is something that _works_, its not the greatest looking thing, but maybe that will improve on a future revision? Since Chris is so look and feel oriented, as well as a technically gifted person, I have become accustomed to being bombarded with interface changes. I have learned a few tricks along the way, and since I was doing some work in the toolkit anyways, I decided to make a wholesale change. I must say it is for the better. I have had nothing but compliments, and I found that I have become so used to thinking about fonts, borders, and shading that I tend to notice all of that stuff more and more. Functionality still comes first, lets not forget our priorities.
I also added the WYSIWYG editor HTMArea into my code base. It's not hooked into the CMS as of yet, but I did make use of it in the toolkit. I did have to chase down a few bugs (relating to IE of course). The editor works great in Mozilla (Firefox), yet on certain versions of IE (windows 2000 version 6.0.28 or something), the actual textarea, to type in, was non existent. This was apparently not a problem on Windows XP with SP1. Apparently the problems on certain versions of IE are when the editor is within nested tables. IE does not handle the size coordinates properly within nested table cells (resets to 0), so the end result is a textarea created within an iframe of 0 pixels. :-( A simple code hack in the htmlarea.js to set the width and height to 100%, and a border around the textarea, and everything seems to be no problems.
There was a slight issue with mozilla based browsers. You could not copy and paste into the broswer. It violates the security settings, as mozilla does not allow scripts on web pages access to your computers clipboard. GAY! But I understand the reasoning, and potential security problems behind that. In order to fix it, you can install a firefox extension to edit your user files, and place some settings in the user.js tab to allow specific web sites clipboard access. Nifty! And everyone is happy again.
Yesterday was an interesting day for me. Since I re-worked the entire L&Y network to run the VPN using Linksys routers, there have been absolutely no problems that I have heard of. That makes me very happy, considering the problematic VPN setup they used to be running under. That's 5 months and not a peep from anyone. Speed seems to be up, and connections are not flaky, no more faulty hardware to deal with etc.
Telus' entire network in the province went down yesterday apparently. The two outlet offices not in the city both happen to be running on Telus. Oh the joys. I was on my way to the office, and I got the phone call from Andy Henderson about problems with the Lethbridge office, and he informed me about Telus. After quite a few minutes on the phone with John at the remote office I realized what had happened. The wonderful tech support that John had been talking to on the phone about their problems, when they first realized the lack of internet, had him reset the modem, and the router. Standard stupid techy help desk fix all for any problems you can't fix. Reset everything and start all over. John did just what he was told, he held that little reset buton in with a pen for 30 secs and reset the VPN router.
It's not really anyone's fault per se. How was the Telus guy supposed to know it's a specially configured VPN router (how many telus clients make use of that?), and John didn't realize what it really does. Now the router is set to factory defaults, and I have no information I need to reconfigure one to courier down. I can't remotely login because it's not even online anymore without the proper settings. Telus does not have true static IPs, they are all assigned by DHCP and held for your MAC address. This particualr router had a cloned MAC address to make installation easier, and like I know the MAC address of the OpenBSD firewall from 6 months ago? With the patience of John and his ability to play around and relay to me what was happening over the phone, we eventually managed to set up the correct MAC address, and configuration so I could get to it remotely and configure the VPN networks. woot! They were all back and running. No trips to Lethbridge for either Andy, and everyone is happy. John won't ever reset the router in the future. The second office didn't touch a thing, and I just logged into the main VPN router, hit the connect button on that tunnel, and bango, they were all connected again. This network is going to be the death of me I'm sure ;-)
I'm flying out to Victoria for one last trip. Well probably not the last trip, but the last major one for a while (I hope). I had some shirts made for Marshall and the girls.
Since I'm flying out on Sunday I've been organizing things before I go. I finished my company year and, and dropped everything off at the accountants. Thanks to Aaron Seigo for killing a few trees printing out my number crunching for me. Thanks Dude! I've cleaned up the clutter and band uniform parts lying around from Robbie Burns night, and now all I have to do is the dishes, laundry, and maybe run the vacuum over the place.
I got some new threads while I was here this time (like $250 I can't really afford ... but oh well). So I decided to go all out before I left and went back to see Renee at Diva Hair. I got highlights, and a smokin' haircut. Renee is totally awesome and always does a superb job on the hair. I get the "friend discount", and I always make it worth her while. Thanks babe! So I'm stylin', and ready to go again. The wheather is turning cold, so It's a good time to hit the west coast again.
¶ 1:42:00 AM 0 comments
I completely re-worked the entire interface for the L&Y toolkit. Chris Lang has been a huge influence on my artisitc web skills since I have been working with him. It's not that I was unaware of user interface techniques, or that I couldn't accomplish what I wanted. Coder's (especially in the OSS world) tend to work on what interests them. Playing with CSS, and picky little user interface issues like changing the font one size smaller, or the font family to something different, and the background shading, and borders, and positioning etc. etc. etc. is not interesting to me. To me it is menial task. My time is better spent doing what I am good at, which is writing code.
I know that all of those things I mentioned are very, very important. Greatly important on a public website. I am a low level coder however, and what interests me is making the web site functional. It's the microsoft vs linux console scenario. I don't care if it looks pretty, the crappy software doesn't do what I need it to. Hey ... over here is something that _works_, its not the greatest looking thing, but maybe that will improve on a future revision? Since Chris is so look and feel oriented, as well as a technically gifted person, I have become accustomed to being bombarded with interface changes. I have learned a few tricks along the way, and since I was doing some work in the toolkit anyways, I decided to make a wholesale change. I must say it is for the better. I have had nothing but compliments, and I found that I have become so used to thinking about fonts, borders, and shading that I tend to notice all of that stuff more and more. Functionality still comes first, lets not forget our priorities.
I also added the WYSIWYG editor HTMArea into my code base. It's not hooked into the CMS as of yet, but I did make use of it in the toolkit. I did have to chase down a few bugs (relating to IE of course). The editor works great in Mozilla (Firefox), yet on certain versions of IE (windows 2000 version 6.0.28 or something), the actual textarea, to type in, was non existent. This was apparently not a problem on Windows XP with SP1. Apparently the problems on certain versions of IE are when the editor is within nested tables. IE does not handle the size coordinates properly within nested table cells (resets to 0), so the end result is a textarea created within an iframe of 0 pixels. :-( A simple code hack in the htmlarea.js to set the width and height to 100%, and a border around the textarea, and everything seems to be no problems.
There was a slight issue with mozilla based browsers. You could not copy and paste into the broswer. It violates the security settings, as mozilla does not allow scripts on web pages access to your computers clipboard. GAY! But I understand the reasoning, and potential security problems behind that. In order to fix it, you can install a firefox extension to edit your user files, and place some settings in the user.js tab to allow specific web sites clipboard access. Nifty! And everyone is happy again.
Yesterday was an interesting day for me. Since I re-worked the entire L&Y network to run the VPN using Linksys routers, there have been absolutely no problems that I have heard of. That makes me very happy, considering the problematic VPN setup they used to be running under. That's 5 months and not a peep from anyone. Speed seems to be up, and connections are not flaky, no more faulty hardware to deal with etc.
Telus' entire network in the province went down yesterday apparently. The two outlet offices not in the city both happen to be running on Telus. Oh the joys. I was on my way to the office, and I got the phone call from Andy Henderson about problems with the Lethbridge office, and he informed me about Telus. After quite a few minutes on the phone with John at the remote office I realized what had happened. The wonderful tech support that John had been talking to on the phone about their problems, when they first realized the lack of internet, had him reset the modem, and the router. Standard stupid techy help desk fix all for any problems you can't fix. Reset everything and start all over. John did just what he was told, he held that little reset buton in with a pen for 30 secs and reset the VPN router.
It's not really anyone's fault per se. How was the Telus guy supposed to know it's a specially configured VPN router (how many telus clients make use of that?), and John didn't realize what it really does. Now the router is set to factory defaults, and I have no information I need to reconfigure one to courier down. I can't remotely login because it's not even online anymore without the proper settings. Telus does not have true static IPs, they are all assigned by DHCP and held for your MAC address. This particualr router had a cloned MAC address to make installation easier, and like I know the MAC address of the OpenBSD firewall from 6 months ago? With the patience of John and his ability to play around and relay to me what was happening over the phone, we eventually managed to set up the correct MAC address, and configuration so I could get to it remotely and configure the VPN networks. woot! They were all back and running. No trips to Lethbridge for either Andy, and everyone is happy. John won't ever reset the router in the future. The second office didn't touch a thing, and I just logged into the main VPN router, hit the connect button on that tunnel, and bango, they were all connected again. This network is going to be the death of me I'm sure ;-)
I'm flying out to Victoria for one last trip. Well probably not the last trip, but the last major one for a while (I hope). I had some shirts made for Marshall and the girls.
- Karen's shirt is bright YELLOW and it says "MARGARITA PRINCESS"
- Melissa's shirt is GREEN with a white collar and it says "Victoria's Next top Model".
- Marshall's is a white NAVY sleaved shirt saying "FEED ME WHEN I FORGET TO EAT"
Since I'm flying out on Sunday I've been organizing things before I go. I finished my company year and, and dropped everything off at the accountants. Thanks to Aaron Seigo for killing a few trees printing out my number crunching for me. Thanks Dude! I've cleaned up the clutter and band uniform parts lying around from Robbie Burns night, and now all I have to do is the dishes, laundry, and maybe run the vacuum over the place.
I got some new threads while I was here this time (like $250 I can't really afford ... but oh well). So I decided to go all out before I left and went back to see Renee at Diva Hair. I got highlights, and a smokin' haircut. Renee is totally awesome and always does a superb job on the hair. I get the "friend discount", and I always make it worth her while. Thanks babe! So I'm stylin', and ready to go again. The wheather is turning cold, so It's a good time to hit the west coast again.
¶ 1:42:00 AM 0 comments
