header1   header
header
header Register : : Login header
header
connector   connector
menuleft menuright
submenu   submenu
left
import-csv from the web...
Last Post 06 Jul 2008 01:00 PM by Brandon Shell [MVP]. 6 Replies.
Printer Friendly
  •  
  •  
  •  
  •  
  •  
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
glnsizeUser is Offline
Basic Member
Basic Member
Posts:193

--
05 Jul 2008 11:56 AM

    little background my current solution is as follows...

    $web = New-Object System.Net.WebClient
    $web.headers.add("user-agent", "PowerShell Script")
    $web.Downloadstring("http://link.php") | out-file $env:tmp\pscsv.csv
    $obj = Import-Csv $env:tmp\pscsv.csv

    I would really like to remove the extra step of saving the file locally.  An accurate simulation without the web service would be...

     

    # create test object gwmi Win32_service | select name, startname | Export-Csv "$env:tmp\test.csv" #import works... $web = New-Object System.Net.WebClient $web.headers.add("user-agent", "PowerShell Script") $web.Downloadstring("FILE://$env:tmp\test.csv")| out-file $env:tmp\pscsv.csv $obj = Import-Csv $env:tmp\pscsv.csv del $env:tmp\pscsv.csv #Doesn't work... $web = New-Object System.Net.WebClient $web.headers.add("user-agent", "PowerShell Script") $obj = Import-Csv $web.Downloadstring("File://$env:tmp\test.csv")

    <!--[if gte mso 9]> Normal 0 false false false EN-US X-NONE X-NONE <!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1107304683 0 0 159 0;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1610611985 1073750139 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-priority:1; mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin-top:0in; margin-right:0in; margin-bottom:10.0pt; margin-left:0in; mso-pagination:widow-orphan; font-size:11.0pt; mso-bidi-font-size:10.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} .MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; mso-bidi-font-size:10.0pt; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} .MsoPapDefault {mso-style-type:export-only; margin-bottom:10.0pt;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.0in 1.0in 1.0in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> <!--[if gte mso 10]>

    I think my problem is how I am downloading the file. Any help would be appreciated...

     

    ~Glenn

    halr9000User is Offline
    PowerShell MVP, Site Admin
    Advanced Member
    Advanced Member
    Posts:565
    Avatar

    --
    05 Jul 2008 07:29 PM
    Import-CSV is a bit limited in that it only accepts a path to a file which is then read and converted to an object. You cannot directly pass data to it in the way you are attempting.

    btw here's a one-liner you may find useful: $tempFile = [System.IO.Path]::GetTempFileName()
    Community Director, PowerShellCommunity.org
    Co-host, PowerScripting Podcast
    Author, TechProsaic
    glnsizeUser is Offline
    Basic Member
    Basic Member
    Posts:193

    --
    06 Jul 2008 06:37 AM

    <!--[if gte mso 9]> Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

    <!--[if gte mso 10]> I have seen that one liner before... I think on your podcast.  Never thought I would need it, but in this situation that is much cleaner.  That kind of sucks that you can’t directly inject data.  Does Import-clixml have the same limitation?

    Thanks!

    ~Glenn

    <!--[if gte mso 9]> Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

    halr9000User is Offline
    PowerShell MVP, Site Admin
    Advanced Member
    Advanced Member
    Posts:565
    Avatar

    --
    06 Jul 2008 06:57 AM
    Well I think we're running into the definition of the verb "import". This seems to be a consistent usage where the import/export cmdlets like to work with files. OTOH, PS v2 CTP 2 includes a new ConvertFrom-Csv cmdlet which works with live objects.
    Community Director, PowerShellCommunity.org
    Co-host, PowerScripting Podcast
    Author, TechProsaic
    Brandon Shell [MVP]User is Offline
    Basic Member
    Basic Member
    Posts:396
    Avatar

    --
    06 Jul 2008 10:51 AM
    Actually... the *-clixml are meant for serializing and deserializing objects. Import-cliXML expects a VERY specific format.

    If you want to "import" XML you can just type it
    [XML]$myxml = < some data >
    Brandon Shell
    ----------------
    Microsoft Powershell MVP
    https://mvp.support.microsoft.com/profile/Brandon
    Blog: http://www.bsonposh.com
    glnsizeUser is Offline
    Basic Member
    Basic Member
    Posts:193

    --
    06 Jul 2008 11:30 AM

     

    <!--[if gte mso 9]> Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

    <!--[if gte mso 10]> Okay, how about a different question... our web/development teams have many home grown apps.  Lately I have begged borrowed and threatened them to give me some Powershell hooks.  Currently this is being provided in the form of custom csv.  Mainly because it's just so damn easy to create one from scratch. 

    Currently this is giving me the ability to inject data directly from all out web stuff into Powershell, but more importantly keeping it in objects!  My days of parsing text are over!!!  So I guess my real question is short of a full blown web service.  What would be the "best" way of getting this data into Powershell? Also by best I mean most efficient...  I mean my current solution works but I hate having to save a 50 mb file, load it, delete it…

    I love Powershell, but sometimes the overhead is just too much... With that in mind I’m always looking for ways to squeeze milliseconds out of execution times.  That way when, something happens and I need to scale up my script won’t crap it self...This overhead is the only reason I have any bat/VBscripts left in my toolbox.

     

    Brandon Shell [MVP]User is Offline
    Basic Member
    Basic Member
    Posts:396
    Avatar

    --
    06 Jul 2008 01:00 PM
    I would suggest .NET remoting, but I am not sure that is much easier than a web service.
    This overhead is the only reason I have any bat/VBscripts left in my toolbox.

    To be honest... I have yet to find a scenario when Powershell wasn’t faster. You will find with Powershell, you always have a lot of options. It is like the old programming adage.
    Good, Fast, and Cheap. Pick any two but you can't have all three.
    With Powershell... if efficiency is an issue you can spend a little more time with the code and achieve "compiled" code type speed, but it does require that time. Unfortunately most of the samples you see on the web are not performance minded. 
    In your case… you want to stick with Text Streaming Classes and handle the file and objectizing (my term) yourself.
    Brandon Shell
    ----------------
    Microsoft Powershell MVP
    https://mvp.support.microsoft.com/profile/Brandon
    Blog: http://www.bsonposh.com
    You are not authorized to post a reply.


    Active Forums 4.3
    right
    footer   footer
    footer Sponsored by Quest Software • SAPIEN Technologies • Compellent • Microsoft Windows Server 2008 R2 footer
    footer   footer