Bulk copy using Enterprise Library in C#

SqlBulkCopy is use to upload the data very efficiently from application to database in one shot. The best example could be of excel data; where we got the requirement to upload all the excel data into database table. One could think to hit the database row by row and storing each row into table, but this approach is totally wrong. This will lay down the performance of your application as well as your database. The best way is to do by using Bulk copy. This functionality is only limited to Sql database table.

Below is the sample code, which depicts the usage of SqlBulkCopy with Enterprise Library.

 int InsertRecord(ref SqlDatabase db, DbTransaction transaction) {
     //here you can enter your insertion code
     // db.ExecuteNonQuery("command", transaction);
     return 1;
 }
 public void InsertDataInBulk(DataTable dtData) {
     SqlDatabase db = null;
     DbTransaction transaction = null;
     try {
         int recordId = 0;
         DataTable dt = dtData;

         //remove any empty rows
         using(dt = dt.AsEnumerable().Where(row = > !row.ItemArray.All(field = > object.ReferenceEquals(field, DBNull.Value) | field.Equals(string.Empty))).CopyToDataTable()) {

             db = (SqlDatabase) DatabaseFactory.CreateDatabase();

             // insert all the records in bulk into the database.

             using(DbConnection connection = db.CreateConnection()) {
                 connection.Open();

                 // open transaction only in the case where you have to do multiple database operation
                 using(transaction = connection.BeginTransaction()) {

                     //insert the file record into filelog table
                     recordId = InsertRecord(ref db, transaction);
                     if (recordId > 0) {


                         using(SqlBulkCopy bulkCopy = new SqlBulkCopy((SqlConnection) connection, SqlBulkCopyOptions.Default, (SqlTransaction) transaction)) {
                             //if you're unsure of the column sequence in source and destination table then it's better to map the columns

                             SqlBulkCopyColumnMapping mapID = new SqlBulkCopyColumnMapping("ProductID", "ProdID");
                             bulkCopy.ColumnMappings.Add(mapID);

                             SqlBulkCopyColumnMapping mapName = new SqlBulkCopyColumnMapping("Name", "ProdName");
                             bulkCopy.ColumnMappings.Add(mapName);

                             SqlBulkCopyColumnMapping mapMumber = new SqlBulkCopyColumnMapping("ProductNumber", "ProdNum");
                             bulkCopy.ColumnMappings.Add(mapMumber);

                             bulkCopy.DestinationTableName = "YourTableName";
                             bulkCopy.WriteToServer(dt);
                         }
                         transaction.Commit();

                     } else transaction.Rollback();
                 }

             }

         }

     } catch (Exception ex) {
         if (transaction != null) transaction.Rollback();
         //Log your error

     } finally {
         db = null;
     }
 }

Bulkcopy can also be used by using SqlConnection. But the above example is other way around.

Efficient DataTransfer via DataSet (especially over Network) through Webservice in .Net

Whenever we are exposing a webservice with a webmethod of return type DataSet, we usually don't care of what extra information dataset has kept. If we serialize this Dataset in XML; then we can have a look of all the information. So particularly, if we see Dataset sends the schema information also, which sometimes isn't required, and apparently it creates a heavy data transfer specially over network.

Normally, you don't feel the difference if Dataset have one to two tables. But surely, if will boost up the performance. Considering the below example; let's suppose we have one webserivce with a webmethod of return type dataset.

public class MyWebService : System.Web.Services.WebService

    {

        [WebMethod]

        public DataSet GetDataset()

        {

            using (MyDataSet ds = new MyDataSet())

            {

                ds.Tables.Add(CreateTableWithData("Table1"));

                ds.Tables.Add(CreateTableWithData("Table2"));

                ds.Tables.Add(CreateTableWithData("Table3"));

                ds.Tables.Add(CreateTableWithData("Table4"));

                return ds;

            }

        }

        DataTable CreateTableWithData(string tableName)

        {

            using (DataTable dt = new DataTable())

            {

                dt.TableName = tableName;

                dt.Columns.Add("Column1", typeof(string));

                dt.Columns.Add("Column2", typeof(string));

                DataRow drow = dt.NewRow();

                drow[0] = "Test1";

                drow[1] = "Test2";

                dt.Rows.Add(drow);

                return dt;

            }

        }

   }

public class MyDataSet : DataSet

    {

        public override SchemaSerializationMode SchemaSerializationMode

        {

            get { return SchemaSerializationMode.ExcludeSchema  }

        }

    } 

Now if you have noticed, we have created our custom DataSet class (MyDataSet); inherited with DataSet class, with overridden property SchemaSerialiazationMode. Because if you will directly set the property of SchemaSerialiazationMode to ExcludeSchema, it won't allow you to do so. By default it is set to IncludeSchema. Below is the actual property:

public virtual SchemaSerializationMode SchemaSerializationMode {
            get {
                return SchemaSerializationMode.IncludeSchema;
        }
            set {
                if (value != SchemaSerializationMode.IncludeSchema) {
                    throw ExceptionBuilder.CannotChangeSchemaSerializationMode();
        }
    }
  }

We can check the data by serializing the Dataset into XML, which has the XML with removed schema. Here is the sample code:

MyWebService web = new MyWebService();

using (DataSet ds = web.GetDataset())

{

      XmlSerializer ser = new XmlSerializer(typeof(DataSet));

      TextWriter writer = new StreamWriter(@"C:\xm1.xml");

       ser.Serialize(writer, ds);

       writer.Close();

}

 

This is very useful if we have multiple tables in one dataset. Hence ExcludeSchema does the trick by overwhelming the load when sending dataset via WebSerivce.