Author |
Topic |
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-02 : 08:10:14
|
I am exporting data via the "for xml auto,elements" clause.The data is bcp:ed out to textfiles.The recipients are now complaining that there are no line breaks between the tags.i.e. they want it more "nicely formatted".So my question is basically, is it possible to handle this request nicely from the server side?I basically just do:bcp "select blah,blah from tbl for xml auto,elements" queryout "D:\..." -w -T -t -rAny way to get line breaks???rockmoose |
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2007-04-02 : 08:19:27
|
Try with "-c". quote: Performs the operation using a character data type. This option does not prompt for each field; it uses char as the storage type, without prefixes and with \t (tab character) as the field separator and \n (newline character) as the row terminator.
Strange though that "-r \n" does not work.Peter LarssonHelsingborg, Sweden |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-02 : 09:10:27
|
I use -w, since we have unicode chars in the tables.Also -w -c only put a linefeed after each row.The for xml clause, only outputs one row/stream.select * from(select 'abc' as col1, 1 as col2 union all select 'def', 2) as thetablefor xml auto,elementsWhen exported it looks like this (open in notepad)<thetable><col1>abc</col1><col2>1</col2></thetable><thetable><col1>def</col1><col2>2</col2></thetable> If possible I would want it to look like this:<thetable><col1>abc</col1><col2>1</col2></thetable><thetable><col1>def</col1><col2>2</col2></thetable> Coming to think of it, I don't think it is possible from sql server.Alternative ideas might be to run the files through some utility / parser that transforms them.Any suggestions?rockmoose |
 |
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2007-04-02 : 09:29:03
|
If you replace "><" with ">" + CHAR(13) + CHAR(10) + "<" then?Peter LarssonHelsingborg, Sweden |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-02 : 14:10:18
|
Yeah, there are some possibilities.Just fishing for a quick alternative.Seems to me it will be to ask the recipients to fix their line breaks themselves.After all it is a presentation issue If I am in a nice mood, I might help them.rockmoose |
 |
|
jezemine
Master Smack Fu Yak Hacker
2886 Posts |
Posted - 2007-04-08 : 11:41:21
|
if you write the xml using XmlWriter, you can use the XmlWriterSettings class to specify how you want the indenting and linebreaks to work. See the Indent, IndentChars, NewLineChars properties on that class. to use this class means you have to consume your stream by a .net app though. perhaps you are already doing so? www.elsasoft.org |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-08 : 12:36:47
|
quote: Originally posted by jezemine if you write the xml using XmlWriter, you can use the XmlWriterSettings class to specify how you want the indenting and linebreaks to work. See the Indent, IndentChars, NewLineChars properties on that class. to use this class means you have to consume your stream by a .net app though. perhaps you are already doing so? www.elsasoft.org
No just using bcp and select for xml.Thanks for the suggestion, I'll check the docs for that. It might be just what I need.I was thinking of just hardcoding the 12 selects like so:select '<bla>' + bla + '</bla>' + char(10) + char(13)rockmoose |
 |
|
jezemine
Master Smack Fu Yak Hacker
2886 Posts |
Posted - 2007-04-08 : 14:00:32
|
doing this is fragile:select '<bla>' + bla + '</bla>' + char(10) + char(13)because if your values contain any chars such as &, <, >, it will break your xml. www.elsasoft.org |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-08 : 16:43:56
|
So, using the XmlWriter, is this the way to go about it?relevant code in bold.Notice, using the XmlWriter I found no straightforward way to leverage the FOR XML clause in SQL.XmlWriter xw = XmlWriter.Create(...);SqlConnection = new SqlConnection("...");SqlCommand cmd = cn.CreateCommand();cmd.CommandText = "SELECT col1,col2 FROM tbl";cn.Open();SqlDataReader dr = cmd.ExecuteReader();while(dr.Read()){ for(i=0;i<dr.FieldCount;i++) xw.WriteElementString(dr.GetName(i),dr[i].ToString());}cn.Close(); Not being a xml hacker I just have to ask, I might be missing the obvious PS.Some of the fields are ntext fields, could that cause some possible problem for the XmlWriter approach?rockmoose |
 |
|
jezemine
Master Smack Fu Yak Hacker
2886 Posts |
Posted - 2007-04-08 : 17:43:52
|
Couple points:1. always use the "using" keyword when using objects that implement IDisposable. Otherwise you run the risk of native resource leaks if you throw before Close() or Dispose() is called. It's a PITA, but that's what you have to do in C# if you want your code to be robust against leaks.2. I would use attribute centric style here, with one attribute for each column. that way it's clear which column cells are part of the same row.3. Check for null or else the SqlDataReader will throw if you access a null cell.4. save the FieldCount in a local so you don't need to make a call for each row. minor point but it might make a difference if you have millions of rows.5. added an example of how to specify indenting, encoding, etc with XmlWriterSettingspublic void rockmoose(){ XmlWriterSettings s = new XmlWriterSettings(); s.Indent = true; s.Encoding = Encoding.Unicode; s.OmitXmlDeclaration = false; using (XmlWriter xw = XmlWriter.Create("C:\\foo.xml", s)) using (SqlConnection cn = new SqlConnection("...")) using (SqlCommand cmd = cn.CreateCommand()) { cmd.CommandText = "SELECT col1,col2 FROM tbl"; cn.Open(); using (SqlDataReader dr = cmd.ExecuteReader()) { int fields = dr.FieldCount; xw.WriteStartElement("root"); while (dr.Read()) { xw.WriteStartElement("rockmoose"); for (int i = 0; i < fields; i++) { if (dr.IsDBNull(i)) continue; xw.WriteAttributeString(dr.GetName(i), dr[i].ToString()); } xw.WriteEndElement(); } xw.WriteEndElement(); } }} www.elsasoft.org |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-08 : 18:14:01
|
Thanks Jez!1. I would have (or used try finally) but omitted for brevity.2. It's for a translation firm, and I think we will use the element approach, and have start-end tags for each row. (They are importing the final results into word).3. Obviously :), nulls must always be handled, I was thinking of dr.IsDBNull(i) ? "" : dr[i].ToString();4. Optimization is always good, for this case, writing xml woohahaha ;). (But yes, one must do what one can!)5. Thanks again.I'll post the final masterpiece here.rockmoose |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-09 : 20:04:55
|
Here is the powershell hack in it's gory glory, enjoy:It creates two files, one for the schema metadata, and one for the actual exported data.The connection is SSPI, and uses the local machine as default.Provide 3 part aming in the SQL, as the connection defaults to master database.All this , just to get linebreaks in the XML  ## sample call## .\exportXml "SELECT TOP 10 * FROM master.dbo.sysobjects" "C:\Dev\ps\" "sys.objects"param($select,$path,$file,$instance)if($select -eq $null -or $path -eq $null -or $file -eq $null){"usage:.\x `"SELECT FROM clause`" `"x:\path\`" `"filename`"";exit;}## get the server name if instance not provided, defaults to local machineif($instance -eq $null){ $srv = (gwmi Win32_ComputerSystem | Select-Object "Name").Name; }## initialize variables and resources$cn = new-object system.data.SqlClient.SqlConnection("Data Source=" + $instance + ";Integrated Security=SSPI;Initial Catalog=master");$cmd = new-object system.Data.Sqlclient.SqlCommand($select, $cn);$cmd.CommandType = [System.Data.CommandType]"Text"$s = new-object System.Xml.XmlWriterSettings;$s.Indent = $true;$s.Encoding = [System.Text.Encoding]::Unicode;$xw = [System.Xml.XmlWriter]::Create($path+$file+".xml",$s);## cleanup resourcesfunction cleanup(){ if($cn -ne $null){$cn.Close();} if($xw -ne $null){$xw.Close();}}## generic error traptrap{ $error[0]; cleanup; exit;}## do the work$xw.WriteStartElement($file);$cn.Open()$rdr = $cmd.ExecuteReader();while($rdr.Read()){ $xw.WriteStartElement("row"); $j = $rdr.FieldCount; for($i=0;$i -lt $j;$i++) { $val = ""; if(!$rdr[$i].IsDBNull){ $val = $rdr[$i].ToString(); } $xw.WriteElementString($rdr.GetName($i),$val); #$rdr.GetName($i);$rdr[$i]; } $xw.WriteEndElement();}$xw.WriteEndElement();## create a file with the schema$rdr.GetSchemaTable().WriteXml($path+$file+"schema.xml");cleanup;## To view the file and the schema file, uncomment# "SCHEMA FILE: " + ($path+$file+"schema.xml"); Get-Content ($path+$file+"schema.xml");# "DATA FILE: " + ($path+$file+"schema.xml"); Get-Content ($path+$file+".xml"); rockmoose |
 |
|
jezemine
Master Smack Fu Yak Hacker
2886 Posts |
Posted - 2007-04-09 : 23:14:37
|
cool. I've been meaning to start using monad but keep putting it off. old habits...one other thing i thought of - if any column names have spaces or other non-xml friendly chars in them, this will break (because you can't have a space in an attribute element name).I'm sure you don't have such poorly named columns though.  www.elsasoft.org |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-10 : 04:56:12
|
The XmlWriter by default does some checking and throws an error in that case:Exception calling "WriteElementString" with "2" argument(s): "Invalid name character in 'spa ce'. The ' ' character, hexadecimal value 0x20, cannot be included in a name."At C:\dev\ps\exportXml.ps1:57 char:25+ $xw.WriteElementString( <<<< $rdr.GetName($i),$val); Yes, I like the powershell (former monad) since you can script with the .NET framework without having to compile stuff into tiny apps.rockmoose |
 |
|
jezemine
Master Smack Fu Yak Hacker
2886 Posts |
Posted - 2007-04-10 : 09:25:24
|
exactly, that's what i meant by "break".  www.elsasoft.org |
 |
|
rockmoose
SQL Natt Alfen
3279 Posts |
Posted - 2007-04-10 : 13:22:28
|
quote: Originally posted by jezemine exactly, that's what i meant by "break".  www.elsasoft.org
LOL Crash in a Null Reference, that's broken.Displaying a friendly error message that can be easily fixed, that's not broken |
 |
|
|